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Chapter  1 
INTRODUCTION 

Since  Shnnntwa-  [ J ")  showed,  that  it  is  theoretically  possible 
through  coding  to  achieve  reliable  communication  at  rates  below  chan- 
nel capacity,  much  work  in  the  field  of  coding  theory  was  centered  on 
finding  and  applying  various  suitable  coding  and  decoding  techniques 
for  memoryless  channels.  One  such  technique  is  convolutional  coding, 
first  discusse  d by  Elias  [2  1 in  1955  and  since  shown  to  be  quite  cost 
effective  in  terms  of  error  correcting  capability  provided  for  a given 
equipment  complexity  level.  The  space  shuttle  communications  system 
[3  land  commercial  communication  satellites  [4  "]  are  but  two  of 
many  current  examples  of  the  use  of  convolutional  coding  to  obtain 
performance  gain  in  a practical  digital  communication  link  having 
restrictions  on  available  power. 

JThe  convolutional  encoding  operation  is  deterministic  and  thus 
quite  easily  implemented.  The  decoding  of  convolutional  codes,  on  the 
other  hand,  must  provide  correction  of  random  errors  made  in  trans- 
mission and  thus  is  a design  problem  with  an  underlying  statistical 
nature.  To  approach  such  a problem  one  ideally  would  like  to  apply 
statistical  estimation  theory  techniques  directly  and  thus  derive  an 
optimal  estimator  (decoder).  In  practice  a feasible  decoder  such  as 
that  due  to  Vitcrbi  [ 5 1 may  be  developed  first  and  only  subsequently 
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shown  to  be  an  optimal  estimator,  in  this  case  the  optimal  maximum 
likelihood  data  sequence  estimator  (Forney  [6  "]  ).  However  Massey 
[7  1 effectively  looked  at  a Maximum  Aposteriori  Probability  (MAP) 
error  bit  estimator  and  showed  how  for  the  orthogonal  subset  of  con- 
volutional codes  this  could  be  specialized  to  the  implementable  thres- 
hold decoding  technique. 

In  this  work  an  attempt  is  made  to  first  consider  the  convolutional 
decoding  problem  from  an  optimal  estimation  viewpoint  and  then 
specialize  to  possibly  suboptimal  but  implementable  decoding  tech- 
niques. While  much  work  on  data  estimation  has  appeared  subsequent 
to  the  Viterbi  algorithm,  here  e r r o r estimation  is  stressed  and  thus 
is  more  akin  to  the  original  work  of  Massey  T?-1  . 

Chapter  2 provides  a brief  review  of  convolutional  coding  and 
decoding  for  memoryless  channels  with  emphasis  on  only  those  ele- 
ments needed  later  to  discuss  error  estimation  and  syndrome  decod- 
ing. Provided  is  a qualitative  breakout  of  the  various  possible  decod- 
ing approaches  for  data  and  errors  from  a probabilistic  viewpoint. 

Chapter  3 displays  general  optimal  error  bit  and  error  sequence 
estimation  techniques  both  with  and  without  the  practical  restriction 
on  time  delay  allowable  before  an  error  estimate  must  be  produced. 
The  complementary  relationship  between  MAP  error  decoding  and 


MAP  data  decoding  is  also  noted. 


Chapter  4 then  concentrates  on  one  of  the  optimal  decoding 
approaches  from  Chapter  3,  namely  Optimal  Finite  Delay  MAP  Error 
Bit  decoding.  Two  suboptimal  approximations,  ROMMAP  and  SUB- 
MAP, are  derived  and  their  implementations  discussed. 

Chapter  5 illustrates  how  the  actual  performance  of  a ROMMAP 
decoder  may  be  computed  and  that  of  the  SUBMAP  decoder  both  upper 
and  lower  bounded.  Also  indicated  is  the  relationship  between  SUB- 
MAP decoding  and  conventional  feedback  decoding.  Lastly,  simulation 
results  of  practical  SUBMAP  decoder  implementations  for  several 
convolutional  codes  are  presented  to  quantitatively  illustrate  and 
confirm  previous  theoretical  predictions. 

Chapter  6 qualitatively  summarizes  the  conclusions  that  may  be 
drawn  from  this  investigation  and  suggests  some  continuations  that 


may  be  worthwhile. 


Chapter  2 


CONVOLUTIONAL  CODES  AND  PROBABILISTIC 
DECODING 


In  this  chapter  a brief  introduction  to  convolutional  coding  and 
decoding  is  provided  stressing  only  those  areas  necessary  to  later 
derivations  and  implementation.  The  basic  communication  system 
problem  model  is  first  presented  to  provide  motivation  for  the  use  of 
convolutional  coding.  Next,  various  decoding  approaches  for  both 
data  and  error  estimation  are  qualitatively  reviewed  from  a prob- 
abilistic standpoint  to  show  the  range  of  options  that  could  be  pursued 
for  implementation  of  the  crucial  decoding  step.  Chapter  3 will  further 
consider  the  error  estimation  options  and  derive  appropriate  optimal 
decoding  techniques. 

2.  1 Basic  Communication  System  Model 

The  underlying  communication  system  problem  can  be  simply 
stated  by  referring  to  Figure  2.  1.  Here  a Data  Source  generates 
digital  data  to  be  reliably  transferred  to  a Data  User  through  a real 
communication  channel  which  is  noisy.  The  output  of  the  source  in 
this  work  is  assumed  to  be  a data  sequence  of  symbols  from  GF(q) 
as  is  the  input  to  the  user.  The  noisy  channel  causes  errors  to  be 
introduced  into  the  data  as  presented  to  the  user  with  such  a frequency 
(error  rate)  so  as  to  make  the  received  data  unacceptable.  The  noisy 
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channel  includes  all  the  modulation,  transmission,  and  demodulation 


of  a real  communications  system. 

To  reduce  the  frequency  of  errors  presented  to  the  user,  an  en- 
coder and  decoder  arc  appended  to  the  communication  system  as 
shown  in  Figiwe  2.  2.  To  simplify  analysis,  the  hina  ry  case  of  GF(q) 
GF(2)  is  assumed  and  the  channel  is  modeled  as  an  equivalent  Binary 
Symmetric  Channel  (BSC)  as  shown  in  Figure  2.  3.  This  is  both  a 
simple  and  appropriate  channel  model  for  a system  which  employs 
the  commonly  used  180  degree  Binary  Phase  Shift  Keying  (BPSK) 
modulation  with  hard  decisions  in  the  receiver  and  which  is  corrupted 
by  Additive  White  Gaussian  Noise  (AWGN).  The  specific  relationship 
between  the  BSC  crossover  probability  p and  the  Signal-to-Noise 
Ratio  (SNR)  encountered  at  the  receiver  will  be  given  in  Section  5.  2. 
The  BSC  is  memoryless  in  that  the  symbol  output  at  each  discrete 
time  instant  is  statistically  dependent  only  on  the  corresponding  in- 
put symbol  and  not  on  any  other  input  or  oxitput  symbol  preceding 
or  following.  Thus  the  BSC  is  also  described  as  a PMC  (Discrete 
Memoryless  Channel). 

In  this  work  it  also  is  assumed  that  the  input  data  rate  to  the 
channel  is  so  far  below  the  channel  capacity  that  sufficient  bandwidth 
exists  to  enable  the  coding  gains  predicted  in  later  sections.  This  is 
commonly  called  an  infinite  bandwidth  assumption. 


Included  in  the  "solution"  of  Figure  2.  2 are  as  yet  unspecified 
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encoder  and  decoder  units.  The  function  of  the  encoder  is  to  map 


data  source  output  symbols  into  code  symbols  such  that  BSC  channel 
errors  made  in  transmitting  such  code  symbols  can  sometimes  be 
corrected  by  the  decoder  - thus  decreasing  the  data  error  rate  pre- 
sented to  the  user. 

In  this  work  convolutional  encoding  will  be  assumed  to  implement 
the  encoder  unit.  This  is  a relatively  simple  encoding  method.  The 
more  difficult  problem  of  implementing  a corresponding  decoder  will 
be  investigated  using  several  estimation  approaches. 

2.  2 Convolutional  Encoding 

A convolutional  encoder  is  defined  as  a linear  finite-state  ma- 
chine employing  one  or  more  v - stage  shift  registers  and  having  k 
inputs  at  each  discrete  time  instant  with  n outputs  determined  by  n 
linear  algebraic  function  generators  operating  on  the  present  and 
possible  v prior  inputs. 

The  idea  of  a simple  convolutional  encoder  is  depicted  in  Figure 
2.  4.  where  each  block  of  k binary  bits  from  the  Data  Source  is  en- 
coded into  an  output  block  of  n (n>k)  bits  to  be  transferred  through 
the  BSC  to  the  decoder.  However,  the  output  n-tuple  depends  not  only 
on  the  current  input  k-tuple  (ij^  , . . . , ij^)  but  also  is  constrained  by 
certain  prior  k-tuples  whose  effect  is  still  present  in  the  encoder. 
Thus  the  constraint  length K of  the  particular  encoder  is  defined  as 
the  maximum  number  of  n-tuple  output  bits  which  can  be  influenced 
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by  an  input  k-tuple  bit.  Each  bit  of  the  n-tuple  is  thus  generated  by  a 


linear  combination  of  K or  less  data  bits.  The  rate  R of  the  code  can 
be  approximated  by  R k/n.  That  is,  though  the  channel  is  transfer- 
ring n bits  per  time  period,  the  information  rate  is  only  k bits  per 
time  period.  The  presence  of  these  "excess"  n-k  bits  represents  a 
feature  to  be  exploited  later  by  the  decoder  to  correct  BSC  trans- 
mission errors. 

The  linear  function  generator  relationship  between  input  k-tuples 

(l|^t  ....  l|  S and  output  n-tuples  (c|^ c|n^)  can  in  general  be 

expressed  in  terms  of  coefficients  of  generator  polynomials  in  arbit- 
rary variables  y and  z (Berlekamp  [8  ] ).  Using  notation  similar  to 

that  of  Berlekamp,  the  generator  polynomial  set  for  k inputs  and  n 
outputs 


G 


(j) 


(y.  z)  = 


k 


E 

i = 1 


V 


E 

h=0 


G 


(j) 

i,  h 


h 

z 


(2.  1) 


for  1 s;  j s n 


yields  the  n-tuple  output  digits  (code  digits)  computed  by 


C0)=  E E G (u  forlSjsn. 

t = i h-o  1 h 


(2.  2) 


Here  t is  the  current  time.  The  variable  y is  used  to  index  through 
the  input  k-tuple  positions.  The  variable  z is  used  to  index  through 
information  bits  within  the  constraint  length  of  the  code.  Since  the 
current  k-tuple  (l|^ ij^)  is  available  at  the  encoder  input  but  not 


1 1 


t 


yet  stored,  the  memory  length  v of  the  encoder  specifies  the  maxi- 
mum number  of  prio r input  k-tuples  which  may  affect  a given  cj'^  . 

Although  the  above  notation  is  sufficiently  general  to  be  used  by 
all  rate  k/n  convolutional  codes,  it  is  worthwhile  to  specialize  to  the 
systematic  rate  1/n  code  case.  Thus  with  k 1 and  D substituted  for 
z,  (2.  1)  and  (2.2)  become: 

G{j)(D)  Y.  Gu'  ^ for  lsj^n  (2.3) 


Y C.  ’ I for  1 sj  sn. 

ho  h l-h 


As  an  example,  consider  the  binary,  rate  R k/n  = systematic 
convolutional  encoder  shown  in  Figure  2.  5a  in  terms  of  binary  unit 
delay  (storage)  devices  and  polynomial  coefficient  weights.  The  term 
systematic  here  means  that  one  code  bit,  . is  always  identical  to 

1 1 ' . The  binary  logic  implementation  is  shown  in  Figure  2.  5b.  In 
this  example  K = v + 1 ~ 5. 

The  set  of  generator  polynomials  is  thus: 


G(U(D)  Y G!n  Dh 


-G'1,tG','DtG'1'D2tG'VtC>V 

U 1 L 3 4 

= 1 and 


G(2)D  - * c<2)  Dh 

E h 

h-0 


Figure  2.5a  Encoder  for  K=  5 Rate  1/2  Systematic  Code 


in  the  form  of  additive  binary  bit  errors  e ^ . Thus  at  the  decoder 
the  received  vector  will  be: 


rd)  x (1) 

C + e 
t t 


(2.  10) 


Thus  R^  G>  in  this  binary  case. 

It  is  instructive  at  this  point  to  very  briefly  mention  the  concepts 
of  encode  r state  and  the  encode  r state  t ransition  probability.  The 
state  of  the  encoder  at  time  t,  , is  defined  for  a rate  1/n  by 


(It-r  Tt-2 :t- v) 


(2.  11) 


systematic  code.  Thus  the  encoder  state  can  be  represented  by  a 
binary  number  composed  of  the  values  residing  in  each  of  the  v physi- 
cal storage  elements  of  the  encoder  at  time  t.  The  encoder  state 
transition  probability  at  time  t is 

the  probability  that  the  next  state  of  the  encoder  will  be  j ^ given  that 

the  encoder  is  now  in  state  of  . 

t 

With  such  definitions,  the  operation  of  the  rate  1/n  encoder  can 
be  considered  in  terms  of  a state  diagram  of  2Vstates.  The  encoder 


resides  in  a current  state  J and  will  transition  to  one  of  two  allowed 
successor  states  depending  on  the  value  of  1^.  For  illustrative  pur- 
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1 


poses  the  simple  v = 2,  K = 3 rate  ^ systematic  encoder  of  Figure 
2.  6a  is  considered  from  a state  diagram  point  of  view  in  Figure  2.6b. 

The  output  code  digits  are  deterministic  functions  of  both  the 

current  state  J and  the  input  I . Since  I becomes  the  first  element 

of  ^ , the  output  digits  are  equivalently  deterministic  functions  of 

the  state  pair  ^ . Thus  a conditional  output  probability 

P(C  I , I ) = P(C  t<S  , J ,)  can  be  defined  for  any  C . Obviously  for 
-t  t t '-t  t t+l'  1 — t y 

the  encoder  this  probability  is  either  0 or  1.  However  in  later  discuss- 
ions it  will  be  shown  that  for  another  linear  finite -state  machine, 
namely  the  syndrome  former  portion  of  a decoder,  the  output  prob- 
ability will  be  more  a function  of  certain  random  inputs.  For  now  it  is 
only  worth  noting  that  the  conditional  probability  P(C  /o^,  1^)  is  inde- 
pendent of  all  prior  encoder  states  J.  for  j < t.  Therefore, 

P<£t/"V  ■'i •'t-r  -V  Vi1  - (2- 12» 

Thus  the  encoder  can  be  considered  as  a Markov  source  having  2V 
state s whose  state  transitions  are  determined  by  the  value  of  the  input 
data  1^  through  state  transition  probabilities  PC/ 1+ ^ 7 o^)  and  whose  out- 
puts are  specified  by  output  probabilities  P(C^  ^).  This  encoder 

Markov  property  was  used  productively  by  McAdam  [9]  and  Bahl  et 
al.  [lO]  in  analyzing  encoder  operation.  It  will  be  applied  and  ex- 
panded in  later  sections  of  this  investigation  to  analyze  a slightly  more 
complex  decoder  problem. 


17 


r 


2.  3 Syndrome h 

For  the  moment  reconsider  the  general  k,  n encoder  case.  In 
accordance  with  Massey  [ 7"]  (Theorem  9)  and  Berlekamp  [k  1 oru' 
can  define  syndrome  digit  s as 


Z 


k v 

0)  _0)  _ v'  V' 

• ' & ,?o  G'-h 


(j)  U ) . . . . 

r . for  k<  1 <n.  (2.  13) 


Using  Eq.  (2. 10)  this  can  be  expanded  to: 


CU'  + eU'  - £ Y,  (C^1  + e(*}  ) 

1 ‘ z 1 ,^0  *’h  t_h  ‘-h  (2.14) 


z(j)  ,Jj)  , _(j) 

t 


t 


and  reduced  by  Eq.  (2.2)  to 


Z 


r-  e 


k V 

f-(j)  (f)  ( . . . «. 

y C.  , e , for  k < 1 Sn. 
< t h t-h  • 

l 1 h 0 


(j)  Jj)_  v*  V' 


(2.  IS) 


In  general  usage,  syndromes  are  quantities  computed  from  noisy- 
received  data  but  are  functions  o£errors  only.  Thus  it  is  evident  that 
the  syndrome  digits  above  are  functions  only  of  the  errors  introduced 
by  the  BSC. 

For  the  binary  rate  1/n  systematic  code  case  the  syndrome  digits 


are 


z«>  , £ c. 


t t 


- (j)  (1)  , , . . < 

e . for  k < 1 £ n. 


h 0 


t-h 


(2.  If.) 


A vector  notation  common  in  later  sections  for  rate  1/n  codes  is 


z 


Z 


(2  )■ 


(2.  17) 


t 

For  the  previously  mentioned  K 5 rale  ^ example  of  l*qs.  (2.5) 
and  (2.6)  and  Figure  2.5  there  is  only  one  syndrome  digit  equation: 


z(j)  eU>  f 

t t 


v 

L 

h 0 


„(j)  (1) 

°h  °t-h 


(i.  e.  j 2 onl  y). 


(2.  18) 


T hu  s , 


z<2'  . 


o t 


~(2)  (i) 

G . e 
4 t-4 


(2)  (1)  (1)  , (1)  , „(D 

= et  ' et  f "t-l  +V2  f °.-4 


(2.  19) 


The  actual  computation  of  Eq.  (2.18)  or  Eq.  (2.19)  is  accomplished 

. • (j) 

by  a syndrome  former  whose  physical  inputs  are  received  digits  r( 

1 ^ j «;n  and  whose  outputs  are  syndrome  digits  Z ^ k<  j sn,  However 
since  Eq.  (2.15)  has  shown  that  the  syndrome  digits  are  independent 
of  the  actual  code  bits  transmitted,  the  analysis  of  the  syndrome 
former  can  be  simplified  by  considering  its  inputs  to  be  only  error 
bits  ep\  lsj  in  introduced  by  the  BSC.  For  example.  Figure  2.7a 
shows  the  actual  physical  syndrome  former  implementation  tor  the 
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r 


K 5 rate  \ code  mentioned  previously.  Figure  2.7b  on  the  other  hand 
shows  the  syndrome  former  that  will  be  analyzed  in  lieu  of  (hat  in 
Figure  2.7a.  One  can  also  refer  to  Figure  2.8  which  indicates  just 
where  the  syndrome  former  is  used  in  relation  to  other  components 
of  the  convolutional  code  communicat ion  system. 

Since  Fq.(2.l6)  applies  at  a particular  time  t at  the  decoder,  one 
can  write  a syndrome  equation  set  for  any  code  by  considering  all  t 
values  in  some  span,  t^  St  sit^-  For  the  example  K 5 code,  if  one 
considers  time  values  including  the  current  time  t and  5 prior  time 
instants,  the  following  syndrome  equation  set  is  obtained: 


N 

1 

(21 

et-s 

, HI 

1 C c 

t-8 

, <n 
f e , 

t-6 

. (ll 
t -7 

t 0 i 

(11 
pt  -9 

Zt-4 

e<2> 

t-4 

* 

* 

, e(1)  t 

t -(> 

■ 0 1 

(11 
‘ t-8 

Zt-3  s 

(2) 

e,-3 

■ "I!\ 

te(1;  t 

t - 5 

1 0 1 

(11 
°t  -7 

Zt-2 

e(2’ 

t-2 

, (*» 
t e 

t-2 

(il 

+ e 

t - 3 

. e(n  , 

t-4 

i 0 1 

'-(i; 
t -6 

Zt-1 

(2) 

et-l 

. HI 

f *.-l 

■ -!!i 

i 0 1 

Z 

t 

(2) 

e 

t 

■ 

t e 

t 

* •!!’, 

■ ^ - 

i 0 1 

*!!i 

Note  how  a particular  error  digit  (e.  g.  ej  t 1 influences  a span 
of  syndrome  digits  (in  this  case  Z.  for  t-1  si*.  t-S).  This  will  be 
significant  in  later  decoding  steps  where  it  will  be  convenient  to  define 
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a syndrome  vector  Z as  consisting  of  all  syndromes  on  the  left  hand 
side  of  the  set  of  syndrome  equations  delimited  by  a specified  linn- 
span.  Thus  for  the  above  example  the  syndrome  vector  is 


= (Zt-5’Zt-4’-’ 


••Zt)  = Z(t-5,t)  = Zt-S 


(2. 21) 


where  four  equivalent  notations  which  will  appear  later  are  given. 

The  number  of  bits  in  a given  syndrome  vector  Z will  be  denoted  as 
either  |Z  | or  LSYN  in  following  work.  Note  that  a particular  span 
of  syndrome  equations  such  as  Eq.  (2.20)  specifies  a set  of  error  digits 
on  the  right  hand  side.  It  will  be  convenient  to  define  an  overall  error 
vector,  E' , as  the  total  set  of  error  digits  appearing  in  such  a span  of 
equations.  When  later  discussing  single  error  bit  estimation,  the 
error  vector,  E,  will  be  used  to  represent  the  set  of  such  error  digits 
less  the  particular  bit  to  be  estimated.  The  number  of  error  bits  in 
E_will  be  identified  as  LEN.  Thus  the  number  of  error  bits  in  E'will 
be  LEN  f 1. 

While  on  the  subject  of  syndromes,  it  will  prove  useful  to  later 
work  to  here  define  the  s yndromc  former  state.  S . as  being  the  bi- 
nary vector  represented  by  the  logical  values  present  in  the  physical 
storage  elements  of  the  syndrome  former  at  time  t.  Thus  one  could 
represent  the  operation  of  the  syndrome  former  in  terms  of  a syn- 
drome former  state  diagram  similar  to  that  defined  for  the  encoder. 
For  example  the  K = 5 rate  ! syndrome  former  of  Figure  2.  7 could 
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be  represented  as  a state  diagram  having  2=2  states.  The  syn- 
drome former  state  transitions  are  determined  in  this  example  by  the 
value  of  e|^.  Thus  the  syndrome  former  state  transition  probability 

P(S  /S  1 for  the  two  allowable  transitions  is  just  equal  to  P(e*":  0) 
t+1  t 1 

and  P(e|^  1)  respectively. 

However  the  syndrome  former  output  probability  is  more  com- 
plex than  in  the  encoder  case  since  the  output— depends  not  only  on 
the  current  state  S and  the,  input  ej1'  but  also  upon  e|2*  (6ee  Figure 
2.7b).  This  complication  will  be  considered  in  detail  in  Chapter  3. 

For  now  it  is  Sufficient  to  note  that  the  conditional  syndrome  former 
* 

output  probability,  PfZ^/S^  S(  ( ^ is  independent  of  all  prior  syndrome 
fornyer  states  S.  for  j < t.  Therefore, 

p<YVsi Sf-W  P,W stn’  <2-22> 

Thus  the  syndrome  former  also  displays  a Markov  property 
analogous  to  that  of  the  encoder.  This  property  will  be  exploited  in 
Chapter  3 to  simplify  analysis  of  optimal  decoding  by  error  estimation 
using  a syndrome  former. 

2. 4 Decoding 

Convolutional  encoding  has  been  shown  in  Section  2.  2 to  be  re- 
latively simple  to  implement.  Decoding  of  convolutional  codes  is 
somewhat  more  difficult.  The  goal  of  a decoder  is  to  provide  to  the 
data  user  an  estimate  of  the  data  source  output  bit  I(  or  sequence 


— (t  ^ ) • Probabilistic  decoding  is  a method  of  doing  so  by  process- 

ing information  available  at  the  decoder  and  outputing  as  an  estimate 
that  value  (or  I_ ) with  the  highest  probability  of  occurrence  con- 
ditioned on  the  information  available  to  the  decoder. 

Since  the  noise  is  additive  (i.  e.  r + E ),  there  are  two 
complementary  approaches  to  such  estimation.  In  direct  data  estima- 
tion  the  most  probable  It  (or  I_)  is  determined  conditioned  on  available 
information  (e.  g.  a received  vector  R ).  An  equivalent  form  of  data 
estimation  first  produces  code  digit  (or  code  sequence)  estimates 
then  deterministically  estimates  corresponding  databits(or  sequences) 
by  solving  Eq.  (2.4).  In  error  estimation  the  most  probable  additive 
transmission  errors  are  estimated  conditioned  on  available  informa- 
tion and  then  subtracted  from  the  appropriate  received  data  to  provide 
code  digit  Ct  or  code  sequence  C estimates.  With  such  error  esti- 
mates. Eq.  (2.4)  then  provides  a means  of  determining  actual  data 
bit  or  sequence  estimates.  Figure  2.  8 is  an  example  of  a case  in 
which  the  available  information  to  the  decoder  consists  of  syndrome 
bits  which  are  used  to  form  the  error  estimate  e 

t ‘ 

Several  probabilistic  approaches  to  bit  and  sequence  decoding 
are  qualitatively  discussed  in  the  next  two  sections  to  show  the  dis- 
tinction between  data  estimation  and  error  estimation.  Error  approa- 
ches explored  in  this  work  are  noted. 

j 
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2.4.  1 Data  Estimation 


A number  of  researchers  have  investigated  optimal  approaches 

to  decoding  convolutional  codes  by  data  estimation.  In  (1)  through  (4) 

below  their  goal  was  to  estimate  the  encoded  information  bit  1 ^ or 

sequence  I,,  by  processing  the  entire  finite  received  sequence 
\ 1 ^ ) 

R(1,N  tv)  ' 

(1)  Conventional  Viterbi  Decoding  selects  the  information  sequence 

A 

I,,  to  be  the  information  sequence  which  maximizes 
(l.N) 

P(WVn  -tv)’ 

where  equilikely  input  sequences  are  assumed.  Consequently,  this 
approach  is  Maximum  Likelihood  Sequence  Decoding.  (Viterbi  [5  1 ) . 

A 

(2)  Map  Sequence  Decoding  selects  the  information  sequence  I,, 

(1,  N) 

to  he  the  information  sequence  which  maximizes 


Vi  / R \ Jic_n  IV  J_ 

V1>NI  ,l>Ntv7 


where  nonequilikely  input  sequences  are  considered.  If  the  input 
sequences  are  equilikely.  conventional  Viterbi  decoding  is  obtained 
(McAdam  [9 1 ) . 


(3)  Map  Bit  Decoding  selects  the  information  bit  I to  be  that  informa- 
tion bit  which  maximizes 


PVR(1,N  tv)1  = ri'',R('-N 

(R  (l.N  tv)  ) 


(Me  A dam  [g*| 

(4)  Maximum  Likelihood  (M L)  Bit  Decoding  is  the  special  case  of  map 
bit  decoding  when  information  bits  are  equilikely. 

Since  it  may  not  be  possible  or  desirable  in  practice  to  process 
the  entire  received  sequence,  as  required  for  the  optimal  approaches 
(1)  through  (4)  above,  some  work  has  appeared  in  which  a finite  stor- 
age length  of  the  possibly  infinite  length  received  sequence  was  used 

A A 

to  estimate  either  an  information  bit  I (or  I ) or  information  se- 
A 


quence  I,  , . This  "real  time  constraint"  on  the  information  bit/se- 
(l.t) 

quence  estimation  has  been  considered  by  the  following  two  approaches. 


(5)  Real  Time  Vite rbi  Sequence  Decoding  seeks  that  information 
A 

sequence  I,,  . , which  maximizes 

M (1,  t +M 


P(I(1,  t+M/R(l,  t + A)  ’ ^d  sets  \ - t-th  bit  of 


sequence  I(1>  t + A) 


This  is  a finite  delay  version  of  the  conventional  Viterbi  Algorithm 


for  equilikely  input  sources  (Lee  [ill  ) . 


(6)  Real  Time  Minimum  Bit  Error  (RTMBEP)  Decoding  seeks  that 


information  bit  I which  maximizes 


P(It  / R 


(1.  t + A) 


) - 


This  is  a finite  delay  version  of  McAdam  Map  Bit  Decoding,  special- 
ized to  equilikely  input  bits.  (Lee  [ill  ) . 
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('-B-  R(l.  1+A)’- 

2.  4.  2 Error  Estimation 

Instead  of  estimating  the  information  bit/sequence  as  above,  one 
might  consider  estimating  error  bits  e ^ or  sequences  e^  ^ ^ and  ob- 

taining an  estimate  of  the  information  bits  by  subtracting  the  estimated 
error  bits  from  the  appropriate  received  bits.  In  such  an  approach, 
a Syndrome  Former  could  be  used  to  produce  syndrome  bits  Z^  from 
the  received  vector  , thereby  making  available  to  the  decoder  a 

vector  Z which  is  independent  of  the  information  bits  or  sequences 

“(1.  t)  M 

actually  sent  and  dependent  only  on  the  channel  errors  which  have 
occurred.  Then  an  optimal  estimate  of  the  error  bit  or  sequence 
e ^ could  be  derived  analogous  to  the  approaches  taken  in  the  case 
of  information  bil/sequence  estimation. 

One  complicating  factor  in  such  an  error  estimation  approach  is 
that  channel  errors  are  not  equilikely  (otherwise  it  would  be  senseless 
to  try  to  transmit  information  over  such  a channel).  Therefore,  Maxi- 
mum Likelihood  (ML)  estimation  must  not  be  used.  Rather,  a Maxi- 
mum A Posteriori  Probability  (MAP)  estimate  should  be  implemented 
which  takes  into  account  the  nonequilikely  apriori  probabilities  of 

channel  error  values  which  are  related  to  the  E /N  level  in  the 
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applicable  channel. 


Another  source  of  concern  in  the  error  estimation  approach 
would  be  the  number  of  error  bits  or  sequences  to  (potentially)  be 
estimated.  For  example,  in  conventional  information  bit  decoding 
of  a rate  1/3  convolutional  code,  only  an  estimate  of  a single  informa- 
tion bit  (e.g.  1^)  is  required.  In  error  estimation  for  this  code,  on  the 
surface  it  might  seem  necessary  to  consider  estimating  as  many  as  _3 
e r ror  bits.  It  will  be  seen  that  it  is  not  always  necessary  to  estimate 
all  error  bits.  Specifically,  in  a rate  1/3  s ystema  tic  code  it  is  obvi- 
ous that  only  a single  error  bit  estimate  is  required.  The  error  esti- 


mation approaches  in  this  work  have  been  derived  for  rate  1/n  sys- 
tematic codes.  However,  the  extension  of  such  probabilistic  decoding 
to  non-systematic  codes  is  relatively  straightforward  although  this 
requires  estimation  of  more  than  a single  error  bit. 

In  this  investigation  the  error  estimation  approach  has  been  pur- 
sued and  the  following  four  estimates  have  been  derived. 

(1)  Map  Error  Bit  Estimate  Decoding  which  selects  as  the  error  bit 
estimate  that  error  bit  value  which  maximizes 


P<VZ,..N+v)'- 

This  decoding  approach  is  basically  an  adaptation  of  the  approach  used 
by  McAdam  [9  1 and  Bahl  et  al  [lO'jfor  MAP  data  bit  estimation. 


U I 


J 


A 

e 


which  maximizes 


1)  Map 


Decoding  which  selects  that  error  sequence 


P(e(l,  N)7  Z(l,  N +v)  } ' 

This  approach  can  be  viewed  in  general  as  an  adaptation  of  Viter- 
bi  decoding  to  the  nonequilikely  error  sequence  decoding  case.  Also 
pursued  is  a conversion  of  a specific  rate  1/3  systematic  code  error 
sequence  decoding  problem  to  a rate  1/2  non- systematic  decoding 
configuration  in  which  the  minimum  Hamming  weight  error  sequence 
approach  of  Schalkwijk  and  Vinck[l3  ]is  useful. 

Approaches  (1)  and  (2)  assume  the  availability  of  the  entire 


received  sequence  R , by  assuming  the  availability  of  the  en- 

M (1,  N +V)  b y 

tire  syndrome  vector  ^ ^ in  (1)  and  of  essentially  infinite  stor- 

age in  (2).  These  are  developed  in  detail  in  Chapter  3. 

Approaches  (3)  and  (4)  deal  with  the  potentially  more  practical 
case  of  finite  delay-finite  storage  (i.  e.  "real  time")  MAP  error  de- 


coding. 


(3)  Real  Time  MAP  Error  Bit  Decoding  which  selects  that  error  bit 


value  e which  maximizes 
t 


pVzo.  ,+4,  >• 


This  is  essentially  an  adaptation  for  error  estimation  of  Lee's 
RTMBEP  Decoding  of  information  bits  [ll  ] . 
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(41  Real  Time  Viterhi  M A P Error  Sequence  Decoding  which  selects 

that  error  sequence  e , which  maximizes 

(1,  t t A) 


P"u.  ltJ0,Z(l.  .*4)' 

and  selects  e as  the  t-th  l)il  of  this  sequence.  This  is  basically  an 
adaptation  for  error  estimation  of  Lee's  Real  Time*  Viterhi  Decoding 


.[111  Approaches  (.1)  and  (4)  are  developed  in  detail  in  Chapter  3. 

Note  that  all  of  the  above  error  estimation  approaches  have  been 


MAP  estimates.  This  is  due  to  the  nonequilikel y nature  of  the  channel 


e rrors. 


After  considering  the  above  "optimal"  approaches  to  MAP  e r ror 


estimation,  a suhoptimal  method  of  MAP  error  estimation  has  been 
derived  in  Chapter  4 which  eliminates  most  active  computations  dur- 
ing decoding.  This  ROMMAP  approach  selects  as  the  error  bit  esti- 
mate e that  error  bit  value  which  maximizes 
t-  A 

P(VS  ' z(,  -s,  n 1 • 

(Basically,  this  is  a finite  delay  ( A ) - finite  storage  (L)  MAP  esti- 
mate). 

The  ROMMAP  is  a suhoptimal  attempt  at  implementing  finite  de- 
lay-finite storage  ("real  time")  MAP  Error  Bit  Decoding  (see  (3) 
above)  since,  to  obtain  a readily  computable  ROMMAP  look-up  table, 
that  portion  of  the  total  syndrome  vector  7.  from  initialization  to 

\ * » W 
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time  t - L - 1 (i.  e.  Z 


(1,  t - L - 1) 


the  ROMMAP  performance  should  be  inferior  to  the  optimal  real 
time. 
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Chapter  3 


WL 


OPTIMAL  ERROR  ESTIMATION 


In  this  chapter  optimal  decoding  of  convolutional  codes  via 
error  estimation  is  explored  in  some  depth.  Several  estimation 
techniques  previously  used  only  for  data  estimation  arc  adapted  and 
applied  to  the  problem  of  error  estimation  based  on  syndromes. 
First  a recursive  MAP  error  bit  estimation  technique  is  developed 


for  the  case  in  which  the  entire  syndrome  vector  Z , is 

— ( 1 . T ) 


available  (i.e.  no  finite  delay  constraint  exists).  Next,  a MAP 


error  sequence  estimator  with  some  analogy  to  the  Viterbi  maximum 


likelihood  sequence  decoding  algorithm  is  analyzed.  Then  a some- 
what more  realistic  case  of  optimal  MAP  error  bi_t  decoding  wi(h 

a finite  delay  constraint  is  derived  in  detail.  For  complete- 
ness, a corresponding  finite  delay  Viterbi  MAP  error  sequence 
technique  is  exhibited.  Finally,  the  relationship  between  MAP  error 
bit  decoding  using  a syndrome  Z_  and  MAP  data  bit  decoding  using 
the  received  vector  R_  is  locked  at  from  a conceptual  viewpoint.  This 
chapter  contains  the  theoretically  optimal  approaches  to  error 
estimation  which  serve  as  the  base  from  which  feasible  suboptimal 
implementations  of  error  decoders  are  obtained  in  Chapter  4. 

3 . 1 Recursive  MAT’  Error  13 it  Estimation 

To  implement  a MAP  estimate  of  an  error  bit  e given  an 
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available  syndrome  vector  _Z  one  would  normally  seek  to  select  that 

A 

value  e^  of  e^  so  as  to  maximize 


P(et/Z) 


P(Z/et)P(et) 

P(Z) 


(3.1) 


Suppose  one  attempts  to  do  this  via  a recursive  estimate  of  e|^  given 
the  total  syndrome  vector  Z_  produced  as  a result  of  syndrome  bit 
computation  based  on  the  total  vector  of  received  bits  R represent- 
ing an  input  information  stream  of  length  T followed  by  v zeros  to 
drive  the  convolut ionnl  encoder  back  to  the  all  zero  slate.  We  can  do 
so  by  using  the  fact  that  the  syndrome  former  is  a linear  sequential 
circuit  whose  output  Z depends  only  on  the  current  syndrome 
former  physical  state  and  the  incoming  error  bits  at  time  t, 
namely  Here  T = T + v . 

Unlike  the  encoder  itself,  the  syndrome  former  does  not 
necessarily  store  all  input  error  bits  in  E^.  Therefore  in  general 
one  cannot  say  that  the  pair  of  states  , S ^ defined  by  actual 
physical  storage  elements  determine  an  output  Z , For  example,  in 
the  rate  1/2  systematic  code  example  the  error  bit  e^  contributes 
to  the  syndrome  output  Z but  does  not  appear  in  any  component  of 
the  physical  state  S represented  by  the  actual  contents  of  the 
storage  elements  in  the  syndrome  former. 

Roth  McAdam  [ g "]  and  Bahl  et  al.  [lO-]  derived  MAP  estimates 


of  information  bits  given  the  total  received  vector  R.  In  the  following, 

3f> 


23 


the  approach  and  notation  of  Bahl  will  be  applied  to  the  derivation  of 
a MAP  estimate  of  error  bits  given  the  total  computed  syndrome 
vector  7.|  . As  usually  done  (e.  g.  Schalkwijk  and  Vinck  [ l 3"] ) we 
consider  the  syndrome  former  inputs  to  be  error  bits  only  since  the 
syndrome  former  output  is  a function  of  these  errors  only  and  not  of 
the  code  bits  which  also  enter  in  R^. 

For  the  sake  of  clarity  we  define  the  following  assumption  and 
later  show  how  although  it  was  true  in  Bahl's  situation  it  does  not 
have  to  be  true  for  the  syndrome  case  at  critical  steps. 

Assumption  If  1 : The  syndrome  former  has  physical  storage  elements 
(whose  contents  we  call  S^)  which  contain  all  components  of  R^.  With 
this  assumption  we  can  treat  the  syndrome  former  as  a Markov 
source  whose  state  pairs  S^,  ^ determine  7,^  and  proceed  as  in 

Bahl's  work.  We  will  note  if  and  when  this  assumption  is  actually 
required  in  the  syndrome  case. 

The  next  figure  shows  a block  diagram  comparison  between 
Bahl's  problem  and  ours; 
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L_ __ A 


Dahl's 

MAP  data  bit 
estimate  case 


MAP  error 
bit  estimate 


•'t 


Goal:  estimate  E^ 


Figure  3.  1 . Comparison  of  Data  and  Error  Estimation 
Approaches. 


We  now  follow  Dahl's  notation  and  estimate  the  a posteriori 

probability  of  the  states  and  transitions  of  a syndrome  former.  In 

V 

the  following  m is  one  of  the  2 possible  binary  values  of  a state. 
Source  Transition  Probabilities 

Syndrome  former  state  transition  probabilities  may  be  expressed 

as; 

pt(m/m')  = Pr(St=m/Stl=m')  ^ 


Our  syndrome  former  state  transition  probabilities  are  governed 


by  the  statistics  of  the  input  n-tuple, 

or 


E(t) 


t 

(2) 


(3.3) 
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through 


Pr(S  -ni/St  l=m')  = ^ PrfS^m,  Et/St_j  =m') 


E. 


Pr(St=m/Et,  St  l=m')Pr(Et/St  ^m') 
E. 


(3.4) 


Since  is  not  a function  of  S^  j=m'  we  have 

Pr  (S^~m/N^  j =m')  = ^ Pr(St=m/Et>  1=m')Pr(Et) 


t3.5) 


Since  the  channel  is  memoryless,  the  individual  bits  in  E are  bit-to- 

n 

bit  independent  and  one  could  let  Pr<E  ) = II  Prlel’V  The  first 

j = l 

factor  on  the  right  is  just  0 or  1 depending  on  whether  the  transition 
is  possible  and  that  the  particular  E^  being  considered  in  the  sum  is 
that  E which  causes  such  a transition  (or  one  of  the  several  E^ 
which  can  cause  the  same  transition  in  the  case  of  some  don't 
care  elements  e|' ' in  E^ ) . 

The  don't  care  elements  can  be  eliminated  from  the  above  sum 
by  defining 

E v = (those  e|^'  which  actually  enter  into  physical  syndrome 
state  storage  elements) 

yielding: 

Pr(St=m/S(  J=m')  = ^ Prps^m/E^ , StJ  =m')Pr(Et' ) . 

Et‘ 


(3.6) 
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Output  Probabilities 

Syndrome  former  output  probabilities  may  be  written  as: 

qt(Z/m',  m)  = Pr(Zt=  Z/St  ^m'.S^m).  (3.7) 

Note  that  qJZ/m1,  m)  = 0 or  1 when  Assumption  #1  is  true  since  j, 
together  contain  exactly  those  error  bits  needed  to  determine  Z^. 

If  Assumption  #1  is  not  true,  then  q^Z/m'.m)  would  not  necessarily 
be  0 or  1 but  would  be  dependent  on  other  error  bits  which  never  re- 
side in  the  physical  state.  However  in  such  a case  one  can  average 
over  these  other  error  bits  to  still  obtain: 

qt(Z/m',m)=  £Pr(Zt=Z/St  j = m',  St=  m,  $)Pr(  g=  g)  (3.8) 

St 

where  <$t  — all  those  error  bits  in  which  affect  the  syndrome  Z^  but 
which  are  not  present  in  the  actual  physical  state  S j,  S^. 


Objective  of  the  Decoder 

The  objective  of  the  decoder  is  to  examine  Zj  and  estimate  the  a 
posteriori  probability  of  the  syndrome  former  states : 


Pr(S=m,  zT  . \ (m) 

Pr(St=m/zJ)=  1 1A  ‘ 


Pr(zJ)  Pr(Zj) 


(3.9) 


and  transitions 


t Pr(s.  , = m',  S = m,  Z?  ) . 

Pr(S.  . = m't  S,=  m /zT  ) = — — =®= 

t_1  1 1 Pr(Zj)  Pr( Z j ) 


m) 


(3.  10) 


Now  derive  the  joint  probabilities  defined  above,  namely, 
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i 


(R  ( 1 , N +v)  ) 
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Here  use  has  been  made  of  the  fact  that: 

(1)  S^=m,  7.^  are  dependent  on  j =m'  but  not  on  Zj  (past 

syndrome  bits)  when  conditioned  on  |>m';  and 
T 

(2)  Zt  ( t when  conditioned  on  is  not  statistically  dependent 

on  j=m'  (past  state),  Z*  * (past  syndrome  bits),  or 

Z (immeidately  past  syndrome  bits). 

Therefore  we  can  write  the  joint  probability  as 

cr  tm'.m)  = Pr(St  ^m'.Z*  1 JPrlS^m,  Z.^/S  =m')Pr(Z{T  j /St=m) 
o (m',  m)  = a i(m')Yt(m',m)Pt(m)  (3.16) 

Forward  Recursion 


Now  for  t = l,2,3,...,T  a forward  recursion  can  be  derived  as 
follows. 

a (m)  = Pr(St=m,  Zj)  = Pr(St  ^m',  St=m,  Zj ) 


all  =m' 


E Pr<st-rm,.st=m.zt1-1.zt) 


all  j =m' 


Y,  Pr(St  ^m',  Z^-1  )Pr(St=m,  Zt/St  l=m',  Zj  ") 
all  S. 

(3.17) 


.t-1 


t-1 


Using  fact  (1)  above  one  obtains; 
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cnannei  error  values  wmcu  are 


it  laau  to  inv  i-  / i'i  icrvti  in  nit 

S O 
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( m ) 


at(m) 


t-1 . 


Y,  Pr(St  j =m't  Z j )Pr{St=m,  Zt/St_ 


=m' ) 


all  S 


t-1 


Y.  at  j (m,)Vt(m,>  m) 
all  S 


(3.18) 


t-1 


At  t=0  we  have  boundary  conditions:  0.^(0)  = = ® f°r 

all  m^O  since  the  syndrome  former  state  is  known  to  be  0. 
Backward  Recursion 

In  a similar  manner  the  backward  recursion  may  be  obtained 
as  follows: 


Pt(m)  = Pr(ZtT+1/St=m)  = £ Pr(St  + l =m’’ Zt  + 1 /St=m' 


all  S 


t + 1 


E Pr(St+rm'>zt+rztT+2/st= 


S,  =m) 


all  S. 


t + 1 


S=m) 


= E Pr<st+rm'-  ^t+i  /st=m,Pr(?:t+2/st+i =m'*  zt+rst= 

an  st+1 

(3.10) 

Again  we  can  state  that  Z^ ^ when  conditioned  on  S^j=m'  is 
statistically  independent  of  Z ^ (past  syndrome  hits)  and  of  St=m 
(past  syndrome  former  state). 


Therefore . 


pt(  m)  = E pt+i(mMYt+i(m'mM 


all  S 


t + 1 


(3.20)  f 
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with  boundary  conditions  3 (0)  = 1,  j3  (m)  = 0 for  all  0 since  we 
have  assumed  that  we  drive  the  syndrome  former  to  0^  state  once  we 
reach  (he  point  in  time  when  it  is  known  that  only  the  tail  of  V zero 
bits  are  being  encoded  at  the  transmitter.  Now 


Yt(m',m)  = Pr(St=m,  Zt/St  j=m') 


= Pr(St=m/St  j=m')Pr(Zt/St  j=m',St=ml 


Yt(m'(m)  = pt(m/m,)qt(Zt/ml,  m) 


(3.211 


Note  that  we  do  not  have  to  average  over  any  intermediate  X since 
in  the  syndrome  case  we  have  jto  Xt  as  was  present  in  Bahl's  problem 
situation. 

Operation  of  Decoder 

The  decoder  must  compute  the  following. 


Xt(m)  = Pr(St=m,  Zj)  = at(m)^(m) 


Syndrome  Former  State 
Joint  Probability 


(3.221 


Ot(m',  m)  = Pr(S(  ^m'.S^m.Zj)  = ^ at  j'm'lY^m'.m) 


st-rm' 


State  Transition  Joint 
Probability 


(3.231 


These  computations  may  be  accomplished  in  the  following  steps: 


Stop  II 1 : 


a0(m),  PT(m)  m = 0,  1 M - 1 are  initialized  (M  = Zv  ) , 
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O.q(0)  = 1 , a.Q(m)  = 0 for  all  m^O 

P (0)  = l,f5  (m)=0  for  all  m?!0 
T T 


Assuming  Initial  and  Final 
States  Known 


(3.24) 


Step  // 2; 


As  soon  as  each  is  obtained  compute 

^(m1,  m)  = pt(m/m')qt(Zt/m',  m)  = PrfS^m,  Zt/St_1=m') 


(3.2  5) 


Next  compute 


at(m)  = ^ at_1(m')Yt(m',  m) 

St-l=m'  (3.26) 

Step  tf  3; 

T 

After  the  complete  sequence  Z has  been  received,  recursively 
compute 


ptim)  = Pt+l(m')Yt+l(m’m,)  (3.27) 

m' 

and  multiply  by  the  stored  at(m)  to  obtain 

Xt(m)  = at(m)Pt(m)  (3.28) 

and  by  Yt(m'  »m)  and  j(m')  to  obtain 

CTt(m',m)  = at_1(m')Yt(m',m)Pt(m).  (3.29) 

Error  Bit  Estimation 

Having  determined  A^(m)  and  o^m',  m)  for  all  0 £ t £ t , now 
estimate  the  required  error  bits  . Assume  that  the  error 
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bits  required  to  appear  explicitly  in  the  syndrome  former  physical 


Let  A^J  be  the  set  of  syndrome  former  states  S such  that 

s|^  = 0 where  is  the  value  of  ej^  after  it  is  shifted  into  the 

associated  syndrome  former  storage  element. 

Then  for  ef^«  E*  where  E*  = [those  ef^  which 

t t t t 

must  be  estimated  and  appear  explicitly  in  syndrome  former  state]  . 
This  implies 


Pr(e|j)  = 0,  Zj)  = £ \t(m) 

c:  a (j ) 


St£At 


(3.30) 


Normalizing  by 


Pr(Zj)  = XT(0)  , 


(3.31) 


the  conditional  probability. 


T 


ym>» 


St£At 


is  obtained.  Now  decode  the  error  bit  by  the  rule: 


(3.32) 


fii  T 

et'J<  = 0 if  Pr(etJ=0/Zj  ) 2 0.  5 


(3.33) 


e^J/  = 1 otherwise. 


Thus  the  MAP  estimate  of  a desired  error  bit  e^'1  given  the 

T 

entire  syndrome  vector  Zj  has  been  obtained  on  a recursive  basis. 
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It  is  seen  that  Assumption  // 1 is  not  needed  for  this  error  bit  esti- 


mation as  long  as  the  error  bit  to  be  estimated  does  appear  in  the 
syndrome  former  physical  stale. 

3. 2 Syndrome  Decoding  of  Rate  1/3  Systematic  Codes 

In  the  case  of  rate  1 /n  systematic  convolutional  codes  the 
syndrome  based  decoder  does  not  have  to  estimate  all  error  bits  in 


(3.341 


The  other  error  bits  e|  , , . . . , e^  can  be  viewed  as  nuisance 

parameters  which  in  MAP  estimation  should  be  averaged  out  in  the 

A(l) 

computation  of  e . This  was  the  approach  followed  in  the  previous 
work  on  recursive  MAP  error  bit  estimation. 

A different  approach  to  the  decoding  problem  exists  in  which 
estimation  of  the  most  probable  error  sequence  is  the  goal  of  the 
decoder.  Recall  that  classical  Vilerbi  Maximum  Likelihood  data 
sequence  decoding  assumes  equilikely  input  data  sequences.  Since 
errors  are  very  nonequilikely,  Appendix  A has  been  provided  to  show 
in  general  how  the  basic  Viterbi  sequence  decoding  metrics  would 
have  to  hi-  modified  to  account  for  nonequilikely  inputs  such  as  error 


sequences. 


Schalkwijk  and  Vinck  [13]  have  derived  and  demonstrated  a 
recursive  algorithm  like  Viterbi's  which  (for  rate  1/2  non-systematic 
codes)  finds  the  error  sequence  of  minimum  Hamming  weight  which 


could  have  caused  the  observed  syndrome  sequence.  This  minimum 
weight  error  sequence  is  the  most  probable  error  sequence  given 
the  observed  syndrome  sequence.  Considering  the  work  that  has  been 
done  [l3]  in  obtaining  minimum  weight  error  sequences,  the  dis- 
cussion that  follows  merely  shows  how  a specific  rate  1/3  systematic 
code  error  sequence  decoding  problem  may  be  converted  to  a rate 
1/2  non-systematic  error  sequence  decoding  problem  suitable  for 
solution  by  the  method  of  Schalkwijk  and  Vinck.  The  rate  1/3 
systematic  code  of  this  example  was  selected  from  Bussgang's  table 
of  optimum  subgenerators  of  a canonical  subgenerator  (Fig.  7 in 
[19]  ). 

Consider  the  syndrome  former  for  a rate  1/3  systematic  code 

shown  in  Figure  3.2a.  The  goal  here  is  to  estimate  the  error 

sequence  (e^  \ e ^ \ . . . , ej^  ^ ) based  on  the  syndrome  sequences 
(2)  (3) 

s'  ,,  and  s'  . . where  N is  the  length  of  the  entire  output 

( 0 , N — 1 ) (0,  N-l ) 

sequence  from  the  data  source.  Figure  3.2a  can  be  redrawn  as  in 


I 

Figure  3.2b  to  more  clearly  isolate  the  "nuisance"  errors  e^  and 
(3) 

e^  . One  can  thus  observe  that  the  problem  of  estimating  the 

desired  error  sequence  e^  \ . . . , e^|  ^ is  equivalent  to  estimating  the 

"data"  input  sequence  to  a rate  1/2  convolutional  "error  encoder"  of 
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Figure  3.2b.  Although  this  appears  to  have  reduced  the  rate  1/3 
problem  to  a possibly  more  manageable  rate  1/2  one,  normal  Viterbi 
sequence  decoding  cannot  be  used  to  estimate  the  error  input.  The 


classical  Viterbi  algorithm  is  a maximum  likelihood  decoder  for 
equilikely  input  sequences  while  here  the  input  is  a very  non- 
equilikely  sequence  of  error  bits  ej  \ 

Now  instead  of  pursuing  a modified  Viterbi  algorithm  with  its 
more  complicated  metric  calculations  for  nonequilikely  error  input 
sequences  (see  Appendix  A),  suppose  error  sequence  decoding  using 
the  method  of  Schalkwijk  and  Vinck  [l3]  is  followed  for  the  case 
shown  in  Figure  3.2b. 

To  keep  things  straight,  call  the  units  of  Fig.  3.2a  and  3,2b 
the  First  Syndrome  Former  (error  encoder).  For  this  error  encoder 
there  exists  a corresponding  Second  Syndrome  Former  shown  in 
Fig.  3.  3a. 


The  operation  of  this  Second  Syndrome  Former  on  input 
syndrome  bits  sj  ^ and  sj  ^ can  be  viewed  as  the  equivalent  operation 
on  error  bits  e|  ' and  e^^  as  shown  in  Fig.  3.3b.  The  rate  1/3 
problem  has  thus  been  converted  to  a rate  1/2  problem  exactly 
like  that  of  Schalkwijk  and  Vinck  [13]  . The  minimum  weight 
(e^\x),  e^(x))  sequence  can  be  found  by  their  technique  and  a non- 


probabilistic  method  used  to  obtain  the  error  sequence  estimate  for 
e|^.  The  Schalkwijk  and  Vinck  method  is  not  affected  by  the  equi- 
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probable  or  nonequiprobable  nature  of  the  encoder  input  e|  ^ since 
their  method  determines  the  minimum  Hamming  weight  noise  sequence 
input  which  due  to  our  assumption  of  a BSC  is  the  most  probable 
sequence . 

The  following  parallels  Schalkwijk  and  Vinck's  explanation  of 
their  approach  as  modified  for  the  rate  1/3  to  rate  1/2  case. 

The  Second  Syndrome  Former  Output  is: 


Z (x)  = C2(x)e(3^(x)  + Cj(x)e'^(x) 

(3.35) 

where 

_ , . .234 

C2(x)  = 1+x  +x  +x 

(3. 36a) 

4 

C3(x)  = 1+x+x 

(3.36b) 

Here  x plays  the 

role  of  the  delay  operator. 

By  Euclid's 

algorithm  there  exists  D_(x)  and  D <x] 

£ J 

such  that 

D2(x)C2(x)  + D3(x)C3(x)  = 1 

(3.37) 

yielding 

D2  (x)  = x + 1 

(3.38a) 

D3(x)  = x. 

(3.38b) 

Received  are 

s(2)(x)  = C2(x)e(1)(x)  + e(2)(x) 

(3.39a) 

s(i)(x)  = C3(x)e(1  ’(x)  + e(3)(x)  . 

(3.39b) 

The  error  sequence  estimate  is  formed  as 


A(l)  (2)  * (2 ) ( 3 \ a / 3 \ 

e'  (x)  = D2(x)(s  (x)  4 e (x))  + D3(x)(s  (x)4e  (x) ).  t3.40) 

«.  (2 ) * / 3 ) 

If  the  noise  sequence  estimate  (e  (x),ev  (x))  is  correct  then 

a(1)  (1)  (2)  a(2) 

e'  '(x)  = D2(x)(C2(x)e  (x)  4 el  (x)  4 e (x)) 

(1)  (3)  -(3) 

+ Dj(x)(Cj(x)e  (x)  4 e (x)  4 e (x)) 

= (D2(x)C2(x)  4 D3(x)C3(x))e(1  \x)  = e(1  '(• 

Equation  (3.40)canbe  written  ns 


a(1)  (2)  (3)  a(2)  a(3) 

e'  '(x)  = (D2(x)s'  '(x)  4 D3(x)s(  '(x))4D2(x)e(  (x)4D3(x)e'  (x) 


(3.42) 


where  with 


(2)  (3) 

u>(x)  = D2(x)e'  (x)  4 D3(x)ev  (x) 


(3.43a) 


A A / 2 ) A ( 3 \ 

UJ(X)  = D2(x)e  (x)  4 Dj(x)e  (x) 


(3.4  3b) 


one  obtains  the  following: 

a / i ) (2)  i 3 \ A 

e'  (x)  = (E>2(x)s  (x)  4 E>3(x)s'  (x))  4 UJ(x)  (3.44) 

■ ' v 

Computed  easily  from  the  Determined  via 

received  s*"^(x)  and  s^(x)  Viterbi-like 

algorithm 

A 

Since  uu(x)  can  be  determined  based  only  on  the  minimum  Hamming 

a (2)  a (3) 

weight  noise  (e  (x),  e (x))  inputs  to  the  Second  Syndrome  Former  and 

not  the  data  inputs  e|  ' to  the  First  Syndrome  Former,  this  approach 

does  not  have  to  be  adjusted  for  nonequipr obable  e|  ' inputs.  Once 

t C 1 


G1  = 

10000 

G2  = 

10111 

1! 

o 

11001 

converts  by  the  above  Second  Syndrome  Former  technique  to  the 


rate  1 /2  nonsystematic  code 


G,  = loin 
g3  = 11001 


given  in  Schalkwijk  and 


Vinck  Table  III,  "Codes  Realized,"  (The  code  is  listed  as  code  4 


with 


g2  = loon 

g3  = 11101 


)•  [Hi. 

From  the  discussion  in  Schalkwijk  and  Vinck  regarding  Table  III 
is  appears  that  had  a rate  1 /3  code  been  used  with 


G.  = 10000 

1 

G! 

= 10000 

G , = 11001 

or 

C,2 

= 11001 

G 3 = 11101 

- C3 

= non 

the  decoding  problem  could  have  "converted"  to  a rate  1 /2  code 
(Codes  3 and  2 respectively  in  Table  III  [ 1 3]  ) whose  decoding  would 
have  required  far  less  complexity  than  the  Code  4 obtained  from  the 
rate  1/3  code  of  this  example. 


*0-th  order  left. 

•:<*0-th  order  right. 
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a*  ■ v*  -m*. 


3.  3 Optimal  Real  Time  MAP  Error  Rit  Estimation 


In  this  section  a recursive  method  for  the  computation  of  the 
optimum  MAP  error  bit  estimate  of  ej  ' is  derived  assuming  a finite 
delay  A.  The  notation  used  throughout  this  section  parallels  that  of 
Fritchman  and  Mixsell  [20]  in  their  work  on  optimal  real  time  data 
bit  estimation. 

Consider  a rate  1 /n  systematic  code.  The  task  is  to  estimate 

one  error  bit  (e.g.  e^  ^ 1 given  the  output  of  the  syndrome  former 

from  the  start  of  the  reception  (i.e.  t=l)  to  the  present  time  t=t 

allowing  only  a finite  delay  A.  That  is,  find  the  maximum  a posteriori 

es'imate  of  e ^ ' given  Z . This  can  be  written  as  e|  ' where 
- ( 1 ) / i \ 

e is  that  value  of  e'  ' that  produces  the 

t-A  t-A  ^ 


L 


maximum 
over  all 
(1 ) 

e . values 
t-A 


I maximum 

= over  all 

(M  i 
e values 
t-A 


i-rie 


t -a’— (i . t r 


Pr(^(i.n> 


13.45) 


where 


(1)  : 


(1)  ^ is  the  error  bit  to  be  estimated: 

(2)  {e|  are  error  bits  which  do  enter  the  syndrome  former 
physical  sta'e; 

(3)  Z, ^ ^ is  the  syndrome  vector  from  startup  to  present 
(containing  all  the  information  possibly  available  to  a 
syndrome  type  decoder);  and 
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(4)  A is  the  "delay"  permitted  before  an  error  bit  decision 

must  be  output  from  the  decoder. 

One  should  note  that  in  (3.451  the  estimate  will  be  made  based 

on  the  entire  vector  Z_  j v'a  a recursive  technique.  This  is  in 

contrast  to  the  ROMMAP  approach  to  be  discussed  later  in  Section 

4.  1,  in  which  the  estimate  will  be  based  on  only  the  finite  length 

syndrome  segment  7.  namely. 

— (t-A.ti  y’ 


maximum 

over  all 

(!)  . 
e values 
t-A 


, *'r*et-A/— (t-A,  t ) 


I maximum 

= over  all 

(1)  . 
e values 
t-A 


Pr(e(1\  Z 1 

t-A  —(t-A.  t) 


Pr(  Z ,1 

— (t  ■ A,  tl 


(3.46) 


For  now,  the  problem  presented  by  Eq.  ( 3.45  1 will  be 


considered.  There  exist  several  methods  of  expressing  the  numer 
ator  joint  probability  Pr(e|  ^)  which  have  been  used  by 

various  authors  [ H]  . [20]  in  estimating  information  bits. 

For  example, 


Pr(e(1),Z,,  ,1  = 

t-A  — ( 1 . t) 


E •••  E 


(D 

(t-A.  t )’  ^— ( 1 , t ) 


...  (1)  , (D  (1) 

withe,.  . . . = { e . . . c 


(1) 

t-A  + 1 


(1) 


t 


(3.47) 


(1 ) (It 

j . e ).  Here  averages  are 


(t-A.  t)  " 1 t-A’  t -A+l  ’ * ' 
taken  over  all  possible  paths  (error  bit  sequences)  of  length  A after 

t lie  particular  error  bit  to  be  estimated. 

By  conditioning  the  numerator  on  the  syndrome  former  state 

^ ^ and  averaging  over  all  possible  values  of  S(  ( ^ one  obtains  an 
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...  • I .iu.' 


i 


f 


1 


/ 1 


values  of  j ' one  can  also  determine  a measure  of  decision 

t-A 

quality  by  summing  (in  the  binary  case)  the  two  numerator  terms 
and  vising: 


A(1 ) 

Pr(  t - A ’■—(!.  t)> 


(Decision  j 
quality  on  = 

Ml)  ) Pr (e)  Z#1  tl)  + Pr(e|  'A,Z,,  AJ 


(3.50) 


t-A 


t - A’  ~( 1 , t ) 


‘t-A’— (1,  t) 


Therefore  since  either  approach  can  be  useful  in  soft  decision 
decoding,  one  can  somewhat  arbitrarily  use  ( 1 ) above  (similar 

to  Fritchman  and  Mixsell  [20]  ) since  this  entails  slightly  less 
computa'ions  and  is  somewhat  easier  to  derive  than  the  approach  (2) 
analogous  to  Lee  [ll]and  [l2  ]. 

In  'he  following,  the  case  of  A > V is  considered.  This  would 
appear  to  be  of  most  applicability  since  one  would  typically  vise  a 
delay  A greater  than  a constraint  length  (v  1 1 ) in  order  to  obtain  as 
much  coding  gain  as  possible  with  a particular  convolutional  code. 
For  completeness,  the  case  A ^ V is  considered  in  Appendix  C 

where  a decision  directed  approach  is  also  described  based  on 

A(1 ) 

incorporation  of  prior  estimates  of  e^  ^ ^ . This  is  similar  to 
Abend  and  Fritchman  [21]  equation  21b. 

Here  let  us  consider  the  numerator  of  (3.3-1) 


Pr(et-a'— mo' 


£ 


Pr'et  -V  — ( i , t r Si+i =s  * 


St+rs 


(3.51) 
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c.  c , (1)  (1) 

S.nce  St  = (et  l,et  2, 


(D  . , „ , (1)  (M 

et-v'andS(  + l =(et  'Vl 

(l)n 


,)  for  this  rate  1 /n  systematic  code  ( where  [e'  are  those 
t-v+1  t 

error  bits  entering  into  the  syndrome  former  physical  state),  we 


can  also  write: 


- E •••  E 


,(D 

't-v+1 


„ , (M  7 (1)  (l) 

Pr(et-A’-(i,t)’et-v+i pn 


(i) 


(3.52) 


or 


(1) 


(1) 


<Me;::,z(lV=  e •••  e 


't-Vfl 


JD 

Jt 


(3.53) 


Now  consider  t lie  Prie * ' , Z , , e| ' ^ , ) terms  of  the 

t -a  — ( t , t ) (t -v+t , 1 1 

numerator  summation  conditioned  on  ej  ^ and  averaged  over  e|  ^ 


Pr|Vy-(i,ii,ct-v4i 


(t-V.  0 


' t-v 


'111 


.) 


tl 


Since  the  statistics  of  /,  , when  conditioned  on  c 


(11 

(t-V,  t) 


(3.54) 

, are  not  a 


function  of  prior  values  of  _Z_  j ^ \ } and  since  the  statistics  of  /. ^ 

when  conditioned  on  e|  ^ , are  not  a function  of  ej  , one  can  write 

(t-v.t)  t - A 


(1) 


(1) 


Pr(-t/e|1-A’-(1.t-1),e(t-v.t))  = Pr(VVv,t)'* 


(3.55) 
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Also,  due  to  the  bit-by-bit  independence  of  the  errors,  one  can 


write 


Pr(e^  7.  ) = 

' t-A  ’ -(l,t-l)’e(t-v,t)' 


Pr(e^)  pr(e^  7 - 

1 t ' 1 et-  A ’ — ( 1 , t-  1) 


’ e(t-v,  t-l)/et  ] ] 


Rut  the  statistics  of  Z,.  , and  of  e/^ 

-(1,1-1)  (t-v,  t-1) 

of  a "future"  e!^  . Therefore 


(3.56) 

are  not  a function 


pr(e^  Z ) =Pr(e^''  ) Pr(e^*'  7 e^  ) 

P t- A ’-(l,t-l)’  G(  t-  V,  t)  ^r(er  ' Pr(et-A’-(l,f-l)>  (t-v,  t-  1)’ 


(3.57) 


and  one  can  thus  write 


Pr(et-A’  -(l,t)>  e(t-v+l,  t) 


"t-v 


Pr(e^)Pr(Zt./e([\t)Pr(e^A,  Z^^e^  ) (3.58) 


This  is  a computable  recursion  since: 

(1)  Pr(e|_A>_Z^j  t-1)’  C(t-v  t-1)^  'S  Pr'  or  num"rator  term  for 
estimating  e|'^  from_Z|2  j. 

(2)  Pr(e|P  ) is  known  for  the  binary  symmetric  channel:  and 

(3)  j)  = Pr(^'{./S| , S t+ 1)  = syndrome  former  output 
probability . 

Since  the  syndronie  former  output  7.  is  a function  of  both  the 
state  pair  (S  , St+I>  and  the  other  "don't  care"  error  bit(s)  (i.e. 
those  that  do  not  enter  into  the  syndrome  former  storage  elements), 
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one  can  obtain  the  syndrome  former  output  probabilities  by  condition- 
ing on  # and  averaging  over  if  ^ where 

' All  those  error  bits  e^' ' in  which  affect  the  syndrome 
= output  Z ^ but  which  are  not  present  in  the  actual  storage 
. elements  representing  (S^,  S j). 


Therefore 


- p'<Vsrsui>  - E Pr(Zt/SCSt+l,lSt)Pr(#tt 


(3.59) 


The  expression  ( 3.53)  for  the  numerator  now  becomes 


P^V-Vo’"  E •••  E P^'t-A’-ll.tr'lt-v+M)' 

i'U  ”1" 


.£  '"5> 

et-vn  t e;_| 


Pr(e|1))Pr(Zt/e[t1_|Vit)). 


•PrK_;,  Zd^.D^t.-v.t-D’ 


(3.60) 


For  notational  convenience  in  the  recursion  explanation  g^  is 


defined  as 


„ 4 pr(e(*)  z e'1)  ) 

gi  ' t-A—  (l.t-A+i)’  (t-A-V+l+i,  t-A+i) 


(3.61) 


By  the  conditioning  and  averaging  discussed  above,  this  is  equivalent 
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7 = V Pr(e  1 . ) P r ( Z / 

’i  t-Ati  — t - At  i 


)Pr(Z  /c^1)  ) 

t-Afi  — t-Ati  (t-A-v+i,  t-A+i) 


ed) 

t-  A + i - \> 

• Pr(e**'  , Z , e^1 ' ) 

t-A’ — (1 , t-A+i-1 )’  (t-A-V+i,  t-A+i-1 ) 


(3.62) 


where 


a = Pr(efl)  Z e(1)  ) 

g0  ' t-A’— (1 , t-A)'  (t-A  -v+1,  t-A) 


(3.63) 


Pr  — (1,  t-A)’  e(t-A-v+l,  t-A)’ 
since  is  contained  in  t_A). 


g = Pr(e**'  Z ) 

t-A’  (M-A+A)’  (t-A-v+l+A,  t-A+A) 


Pr(e(1)  Z e(1)  ) 

( t-A’  (l,t)’  (t-v+l,t)' 


(3.64) 


is  exactly  the  term  needed  in  the  numerator  summation  ( 3.54  ) to 

a (1 ) 

decide  on  e,  . 

t-A 

Now  following  the  modification  of  Fritchman  and  Mixsell  [20] 
to  Lee's  algorithm  to  use  the  minimum  delay  one  can  define 


where 


Q = R = prie*1'  , Z , ) 

i ”i+v  t-A’  (l,t-A+i+v)’  (t-A  +i  + l , t-A+i+v) 


Q0  ~ gv  = Pr(°t-A’  Z(l,t-A+v)’  e(t-A+l,t-A+v)) 


(3.65) 


= Pr(Z.  ,e  ) 

t.t-A+v)  (t-A,  t-A+v) 


(3.66) 
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°A-v  = 8A~  ,>r(°t -A'  /'(l,t),e(t-v+l,t)) 


(3.67) 


which  is  exactly  the  probability  needed  in  ( 3.53  ) to  decide  on 
ej  ^ . That  is  the  numerator  ( 3.45  ) equals 


pO‘-z<M)>  ■ E 

(1) 


Y Pr(e(1,=i,Z.  ,e(1)  , ) 

t-A  ( 1 , t >’  (t-v+1,  t) 

(1) 

et  (3.68) 


which  is  expressible  as 


(1)  (1) 

e.  . e 
t-\»tl  t 


(3.60) 


for  j = 0,  1,  in  this  binary  case. 


Recursion  for  Q. 


Given  for  e^  ^ = j,  i = 0 , 1,  it  is  possible  to  recursively 

compute  Q.,Q., Q as  shown  below.  (In  the  following  the 

i £ A-v 

superscript  of  ej1*  will  be  dropped  to  ease  notation,  i.e.  oj^e  . ) 


!i  = E P(et-S=j'Z,l.t-S+H 


A+i-l  v)’e(t-A+i,t-A+itv)* 


t-A  -t  i 


^ 1 ^'t-A  li+x/1  t-A  •’’ /-( 1 , t-A+i+V-1 )’ * t -A+i.  t -A  t i-l  v) 


t-A+i 


i P(e  =i.  7.  e ) 

t-A  ‘ (1,  t-A+i  Iv-M  (t-A+i,  t-A+i  + v) 


= E r‘Z,.4tH  ' 


A+i  + v (t-A+i.  t-A+i+v  )^  ^ t -A  ( 1 . t - Al  itv-i )’ 


e ) 

(t-A+i.  t-A+i  + v) 


Y 1 <et-A+i+v)P(°t-A  Z(l,  t-A+i+v-D' 


° (t-A+i,  t-A+i+v- 1 )’ 


"t-A+i 


^ t-A+i+v  (t-Afi,  t~A+i+v)^ 


(3.70) 


The  ref ore , 


Q.  = V P(e  . )P(Z^  /e  IQ  , I 

i t-A+i+v  t-A+i+v  (t-A+i,  t-A+Nv)  i-1  I 


, (3.71) 


Note  that  since  both  numerators  (one  for  e =0  and  one  for  e =1) 

t-A  t-A 

must  be  compelled,  at  each  step  in  the  recursion  there  are  two  Q 's. 

i 

Decision 


Once  the  two  parallel  recursions  are  complete  (yielding  Q for 

A-v 

A 

e = 0 and  e =1)  one  can  then  decide  on  e by  comparing  the 
t-A  t-A  t-A 

numerator  for  e = 0 , 
t-A 


P(*.  = 


t-A  ’ (l,t) 


E.  . . V P(e  =0.  Z , , e 

1—.  t-A  (l.t)’  (t-v+1. 


e , e 
t-v+1  t 


= ^ ...V*  Q,  (withe  =0) 

1 Z_ i A-v  t-A 


e , e 
t - vt  1 t 


(3.72) 


with  the  numerator  for  e(  =1  , 


P(et-A=1,/(l,t)'  = Y "'Y  P(ct-A=1, /(l.t)'e(t-vHl,i 

et-v+l  °t 


EQ  (with  e = 1 ) 

A-v  t-A 


(3.73) 


c.  , c 
t-v+1  t 


Determination  of 

The  two  starting  probabilities  an<*  ^0^Ct-A=1’  arC 

needed  to  initiate  the  above  Q-recursion.  These  may  be  obtained 
as  follows: 

Define 

q*e(t-A,  t-A+v  )’  ^0  P^'(l , t-A+v)’ e(t-A,  t-A -K))'  f3-'^ 

and  in  general 

q(e(t-v+i.  t+i))  = P(Z(1,  t+i )’  C(t-v+i.  tH  i ))  ' (3.75) 

This  q function  provides  the  necessary  starting  probabilities  Qq 
in  a manner  similar  to  Lee  [ll]  in  which  a denominator  recursion  was 
used  to  provide  the  start  for  each  numerator  recursion. 

By  using  conditioning  and  averaging  one  can  break  up  the  joint 
probability  for  general  i and  write 

q(c(t-v+i.t+i)'  = P(et+i)P(Zt+i/e(t-v+i,t+i))  H q(l'(t-vti-l,  t+i-1)' 

t -\H i - 1 (3.76) 

Below  is  an  illustration  of  the  computation  of  a specific  case, 
namely  i = v-A  . of  the  determination  of  Q^. 

Q0  = q(e(t-A.t-A+v))  = P(e(t-A,  t-A+v )’  t-A+v)’ 

= P(Zt-A4v/Z(l.t-A+v-D’ert-A,  t-A+v)’1  <Z(l.t-A+v-n,P(t-A.  t-A+v) 
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P(Zt-A+v/,e(t-A.  t-A+v  )”  <Z(1.  t-A+v  -1 )’ 4 (t-A.  t-A  + v)’ 


= P(e.  )P (Z.  .,./ert  ...,,)  £ P(Z 


t-A+v  t-A+v  (t-A,  t-A+v) 


t-A-1 


(1 , t-A+v- M’  e(t-A-l.t-AA>-l) 

■ — - / 


q(C (t-A-1.  t-A+v-1)' 


(3.77) 


Thus  it  is  feasible  to  obtain  Qq  from  the  previous  value  of 

q(e;i  , .,).  One  should  note  that  P(e  ) in  the  above  is 

(t-A- 1 , 1- A+v- 1 ) t-A+v 

known  for  the  binary  symmetric  channel.  The  second  probability, 

P(Z  /e  ) must  be  averaged  over  the  "don't  care"  errors) 

t-A+v  (t-A,  t-A+v) 

namely, 

P(Z  /e  ) = V P(Z  /e  ) 

t-A+v  (t-A,  t-A+v)  t-A+v  (t-A.  t-A+v)  t-A+v 


t-A+v 


■P(*f  .,  ) 
t-A+v 


(3.78) 


where  <?  represents  those  error  bits  which  affect  the  output  of 

t-A+v 

the  syndrome  former  at  time  t-A+v  hut  do  not  appear  in  the  syndrome 

(2)  , 

former  physical  state  (e.g.  e,  , in  the  rate  , example).  This 
r t-A+v 

type  of  computation  of  the  syndrome  former  output  probabilities  was 

to  be  expected  since  this  analysis  is  done  for  a multiple  input 

syndrome  former.  Some  inputs  (e.g.  c|  \ needed  for  decoding  must 

(2) 

be  estimated  while  other  inputs  (e.g.  e^  ) are  nuisance  inputs  which 


must  be  averaged  out. 
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does  not  have  to  be  adjusted  for  nonequiprobable  e{  inputs.  Once 


It  is  best  to  note  that  Q„  = q(e^  , . , , .)  is  actually  a set  of 

0 (t-A,t-A+v) 

2 V * numbers  representing  the  joint  probabilities  P(Z^j  ^ 

e ).  One  can  consider  each  of  these  numbers  to  be 

(t-A.t-A+V) 

labeled  by  the  associated  binary  valued  vectors  (e^  , e^  j, • • • > 

e ).  Thus  the  initial  Q starting  probabilities  for  the  Q 
t-A+v  0 

recursion  to  Q is  a substantial  set  of  2V+  numbers  labeled 
A-v 

by  v+1  binary  error  values. 

However  in  the  algorithm  to  follow,  one  never  has  to  process 

more  than  2V  H 1 numbers  since  in  the  Q recursion  the  contribution 

of  a new  error  value  is  included  while  that  of  an  old  error  value  is 

averaged  out.  Thus  the  necessary  storage  for  2V  + numbers  is 

maintained  throughout  the  recursion  to  Q (A-v  steps).  The  final 

A-v 

V averaging  steps  following  the  Q recursion  reduce  the  2 ^ numbers 

down  to  two  upon  which  the  decision  on  e is  made.  One  does, 

t- A 

however,  need  to  still  retain  the  2N'!  values  for  until  subsequent 

A 

steps  decide  e since  these  values  are  needed  to  compute  the  new 
t -A 

set  of  when  t is  increased  by  one  to  estimate  e^  in  the  next 

pass.  Thus  the  maximum  storage  required  is  the  sum  4 +4  = 


Algorithm  for  Real  Time  MAP  Prror  Tit  Estimate 


Relow  is  given  the  optimal  real  time  MAP  error  bit  estimation 
algorithm  assuming  a delay  of  A.  The  statements  which  are  boxed 
represent  the  actual  algorithm  wilh  the  remainder  being  additional 
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-oO -t H order  right. 


clarifications. 

Step  0:  Set  q(e  ) = 1 for  t = 0. 

I 1 ^ ' 

For  t = 1,2 V+1  calculate 

q(e(l,t))  = Pr(et)pr(zt/e(lft))q<e(itt-l)) 

Note  this  is  the  case  of  Eq.  ( 3.76  ) with  i = 0 and  t Sv+1.  Thus 
no  summation  is  required  untilt  is  >v+l.  That  is 

et-vti-l 

With  i = 0 one  has 

q(e/t  n'  = Prfe  )Pr(Z  /e  ) V q(e  ) 

(t-v,t)  t t (t-v,t)  t—i  (t-v-l,t-l) 

et-v-l  (3.79) 

and  for  t <:  v+1,  q(e(t.v  t))  = clfe(1  t)>-  Thus 

q(e(l,t))  = Pr(et)Pr(Zt/e(l,t))q'e(l>t-l))  f°r  tSv+1- 
Considering  the  above,  the  values  of  the  initial  start  up  q terms 
can  be  tabulated  as  below; 

i=0,t=0:  q(e^  = 1 (one  term)  . 

i=0,t=l;  q(e^  i ))  = PfejJPfZj  /e^  j^qfe^  Q^)  (two  terms) 
this  yields: 


69 


-v  ,**  - . . 


q(c(i  =0)  = PlepO)P(Z1/e1=0)(l|  = Pr(e1--0,Z1) 

q(e  1}=1>  = F»(ei  = 1)p(Zi /ei=1 ) (1)  = Prfe^l.Zj). 

i=0,  t =2 ; q(e(1  2)'  = P(e2)P(Z2/c(l  2)'q(e(l  1)'  (fourterms> 
this  yields: 

q(.(12)=0°l  = P|,2=°lp(Z2/e(i;,r00l„C(iii,=0) 

ii«(i.2r01|  = p,e2=1|p|z2/'(i.2r01,q,c(i.ir01 
qie(i.2r10»  = p"-2'0,p|z2/eli.2r10,q,e(i.ir11 
q|0,i.2r“’ ' p,c2'1’p<z2/oii.2r"'q"(i.ir"  • 


i=0, t=v+' : q(<’(l,v+l)'  * p<ev+l'p(Vl/e(l.v+ll’q('(l.Nl’  U'  ,erm<" 

(3.801 

The  above  computations  form  a tree  that  is  v+1  branches  long 
having  2V  + * terminal  nodes.  This  "initial  error  tree"  is  sketched 
in  Figure  3.5. 


Set  P(e  ,Z  ,,  e Ll  = q(e,f  .. 
t-v  ( 1 , t ) (t-v+l,t)  (t-v.D 

If  Asv  go  to  Step  4 (see  Appendix  C)  . 


Figure  3.5  Optimal  Real  Time  MAP  Error  Bit  Estimate  Initial  Error  Tree 


5R 


Step  2;  (The  Q-recursion  for  A > v case.  ) 

For  i=l,2(.. 

. , A-v 

calculate 

p (e  V e ) 

t-v’  (1,  t+i)’  (t-v  + l+i.t+i)' 

= V P(e  . )P(Z  .A  . t ) 

L-!  t+1  t+1  (t-V  + l.t  + l) 

Ct-v+i 

*P<et-v’  Z(l,t+i-  1)’  e(t-v  + 

i,  t+i-  1 ) ^ 

Step  2 basically  increments  by  the  new  error  bit  and  averages 
over  an  old  error  bit.  Specifically  for  t = \)+l  this  becomes; 

i=l,  Q. : 

i 


Pie  Z e ) 

1 (1,  v+2)’  C(3,  v+2) 


= E p<%+2'p<z +2/e 


'v+2’"  '"v+2/e(2,v+2)1P(el'  Z(l,v+l)’e(2,  v+l)} 


E P(ev+2)P(Zv+2/e(2,vt2))q(e(l,v+l)) 


2V^  of  these  terms 


(3.82) 


It  is  instructive  to  consider  just  what  the  above  Qj  calculation 


actually  involves.  Recall  that  q(e  , is  a set  of  2 

(l,v+l) 


v+1 


numbers 

(probabilities ) each  labeled  by  values  of  (e . , e ) as 

1 i.  v41 

indicated  in  the  initial  error  tree  of  Figure  3.5  . The  step  2 for 

^=0.  Next  one 
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requires  averaging  over  e?.  Therefore  one  sets  e 


e.  ).  These  probabilities  are 
1 +A 


(e  1 ’ e A -v+2’ 


p(el’  Z(l,  l+A)’e(A-v+2,  A + l)*  * 


Now  Step  3 is  entered  to  average  out  e through  e 

A-v+2  A+1 


Step  3:  (Final  average  and  decision  for  A>  V case.  ) For  i=0,  1 
calculate 


A.  = 

i 


If 


E '"eE 

t+A-2v+l  t+A-v 


( 0 7.  e ) 

t-v  ’ (1,  t + A-v)’  (t+A-2\)fl,  t+A-v) 


A >A,  decide  e^  =0,  otherwise  e =1.  Go  to  Step  5 to 
0 1 t-v  t-v 

"increment"  q, 

A 

For  estimating  with  t = v+1  these  quantities  are 


*0=  E --E  Plei=0'z(., 


4+1 )'  1 4 —„'+^ . 4+1 ) 


e t e . 
A~vf2  A+l 


(3.84) 


A1  = £ *’*52  P(el=1’ Z(l,  A+l)’e(A-v+2(A+l)' 

6A-v+2  eA+l 

Again  it  is  instructive  to  state  just  what  is  being  done  here. 

For  A-  - elect  those  numbe  rs  of  the  set  Q = fp  (e  . , Z , , ... 

0 A-v  1 (l.A+l) 

e _ . .)]  labeled  by  e =0.  Call  this  subset  Q (e  =0).  It 

(A-vf-2,  A+1 ) 1 A-v  1 

contains  2V  numbers.  At  the  first  summation  over  e select 

A -vt-2 

those  numbers  in  Q (e  =0)  whose  labels  match  in  all  but  the 
A-v  I 

e position 

A-v+2  r 
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P(er0,Z(l,  A'*  1 )’  e ' &-v*  2 =0 A-fl)1 


P(e  -o  /.  e 1 

' 1 ’ (1 . A-t  1 )'  (A-\H2  1 &fl) 


and  add  such  pairs  equally  weighted  to  obtain  2 elements  in  t lie 
set  of  new  probabilities.  Repeat  this  process  a total  of  v times 
resulting  in 


A0  = P(er°’Z(1,Ml)) 


Ai  = p<tr1'z(i.Afi.) 


If  Aq  > Aj  decide  0^=0,  otherwise  e =1. 


Step  5;  (Increment  q.  ) Increase  t by  1 and  calculate 


l-v-1 


Co  to  Step  1 . 


Here  for  example  t = v+1  would  increase  to  l - 2 to  give 


(2,  vt-2)^  P(ev+2)P(Zvt2/e(2,  v+2))  XI  q(c(l.v+U) 


(3.8M 


Here  for  each  of  the  2 numbers  q(e,.  . ) in  the  set  from  the 

( 1 . V+1) 

initial  error  tree,  find  the  pair  q(e  =0,  e, e ), 

1 2 yl  1 

q(e,=l,e,,.  . . , e ) whose  labels  differ  only  in  the  e.  position  and 

1 2 y|  1 1 

add.  This  yields  2V  new  numbers.  Next  produce  2N*  numbers  by 


multiplying  each  by  P^OJP^/e^ and  hy 

P(e  ~ = 1)P(Z  ~/e,~  . » . ) * This  gives  a new  effective 

\^t  • • • > \H  1 , ) 

error  tree  of  2V*  values  of  q(e  0 9 ) for  use  in  deciding  e during 

( ^ » v* £ ) £ 

A 

the  next  pass.  Now  go  to  Step  1 to  process  for  . etc. 

Thus  the  complete  algorithm  for  estimating  the  error  bit  e^  ^ 
at  time  t has  been  stated.  This  algorithm  represents  the  optimal 
approach  since  the  complete  history  of  syndromes,  namely  Z 


has  been  used  to  make  the  MAP  error  bit  estimate  with  a finite 


delay  A.  This  approach  should  provide  performance  the  same  as 
that  of  a MAP  estimation  of  information  bits.  However  the  actual 


implementation  is  complicated  by  the  non-equilikely  nature  of  the 
error  bits.  This  requires  use  of  actual  probabilities  or  logarithms 
of  probabilities  in  the  error  bit  case  as  opposed  to  the  simpler 
Hamming  distance  metric  useable  in  a Viterbi  decoder  for  inform- 
ation bit  sequences.  With  sue  i a drawback  due  to  the  associated 
computation  overhead,  one  is  tempted  to  investigate  possible 
suboptimal  alternatives  to  the  above  using  a finite  length  syndrome 

vector  such  as  '/,  . This  will  be  done  in  Chapter  4. 

(t  “A.  ' 1 

The  algorithm  given  above  is  complete  for  the  case  \>  v.  For 
the  case  A s v the  reader  is  referred  to  Appendix  C which  gives  the 
missing  Step  4 needed  if  S<.\)  . 


7h 


1 


3.4  Real  Ti me  Viterbi  MAP  Lrror  Sequence  Decoding 

Conventional  Viterbi  data  sequence  decoding  of  convolutional 
codes  lias  been  shown  by  Viterbi  [l4"|  and  Heller  and  Jacobs  [l7l 
to  be  a very  attractive  approach  for  the  equilikely  data  situation. 

In  such  a case,  the  Viterbi  algorithm  has  been  shown  by  Viterbi  [i  ll 

A 

to  choose  I , to  be  the  information  (data)  sequence  which  maxi- 
(1 , N) 

mizes  the  conditional  probability  Pr(L.  ./R  , .)•  When  all 

( 1 , N 1 — ( 1 , N 4 v ) 

input  data  sequences  I ^ are  equilikely,  significant  simplification 
of  the  metric  computations  is  possible.  Indeed  MAP  and  Maximum 
Likelihood  estimators  have  the  same  form. 

As  another  example  of  how  conventional  decoding  algorithms 
adapt  to  error  decoding,  the  conventional  Viterbi  approach  will  now 
be  extended  to  error  sequence  estimation.  This  will  require  incor- 
poration of  the  non-equilike  ly  nature  of  the  error  inputs  to  the 
syndrome  former  and  will  assume  a decoding  delay  requirement 
of  A*  Recall  that  for  a BSC,  higher  weight  error  sequences  are  much 
less  probable  than  low  weight  error  sequences.  Another  complica- 
tion is  that,  unlike  the  information  sequence  case,  not  all  error  bits 

(2) 

enter  into  the  syndrome  former  physical  state.  Some  e.g.  o in 
the  rate  [ systematic  case)  modify  the  syndrome  former  output  /.  but 
never  actually  residue  in  a syndrome  former  storage  element. 

Making  these  two  adjustments,  the  Real-  Time  Viterbi  Decoding 
Algorithm  of  Lee  [ 1 l”|  is  rederived  below  for  error  sequences. 


^.1 


For  a syndrome  decoder  with  delay  A . the  Viterbi  Decoding  Rule 


for  Error  Sequences  would  be: 

*(  1 ) 

Choose  as  the  first  bit  of  the  syndrome  former  state 

S , in  the  state  sequence  S,,  . , , which  maximizes 

t+1  ( 1 , t+A+1 ) 

the  conditional  probability 


P’|S|1.H»1),Z|M)1)1 


|,r|S[i  .11  >nr  'n.mi1 

Pr,z(i.t+a’ 


( 3. fib) 


As  in  previous  work  the  denominator  is  the  same  for  all 
candidate  state  sequences.  Thus  the  necessary  maximization 
can  be  done  on  the  joint  probability  in  the  numerator. 

The  syndrome  former  decoder  for  the  K=5  code  with  finite 
delay  is  shown  again  in  part  in  Figure  3.6  to  clarify  state  and 
error  bit  relationships. 

Recursion  for  Joint  Frobahility 

The  numerator  joint  probability  may  be  converted  to  a recursive 
form  as  follows; 


Pr(S(i,tnr  z«i.t))  = Pr(S 


s /,  . / 

(i.tv  t-tr  (i.t-ir  t 


= 1’riS  /,  ll’rtS  , , 7 /S  ./  1 

' (l.tT  (l.t-1)  t + 1 t (1.  t)  (l.t-11 


” Pr|S(l.D'  Z[l,l-l|1P'|SHl-  W 
J{ l . t+ 1 )"  "(l.n1  = pr(S(lt  o’  7i l 


New  Joint  Probability  Old  Joint  l’rob-  State  Irans-  Syndrome 

ability  it  ion  Prob-  Former 

ability  Output 

Probability 
(3.871  ?8 


In  the  above,  the  fact  that  the  joint  probability  of  S ^ and  Z^,  when 


RatluT  they  are  determined  by  the  cliannel  error  probability  as 


represented  by  E ' . 
Initial  Conditions 


If  it  is  assumed  that  the  syndrome  former  physical  state  at  1 = 1 
is  0,  then  the  initial  conditions  for  the  above  recursion  are: 


Pr(S(l,2)’  Z(l,  1)' 


(1)  rr(S2/0)Pr(Z1/0>  S2) 


if  C)e{<lS2)  and 
0 if  (W^(S2) 


(3.90) 


Pr(S2/0)  = y*  Pr(S2/Ej.0)Pr(Ej)  , 


(3.91) 


^(S. ) = the  allowed  predecessor  states  which  could 

i 

transition  to  S.  , 

i 


Pr(Z1/0,S2)  = £ Pr(Z1/0,  S2,^)Pr(5t)  . 


(3.02) 


The  best  sequence  S 


( 1 , t + 1 ) 


for  maximizing  Pr(S 


( 1 , 1 4 1 ) ’ / (1  .t)' 


at  time  1 4 1 with  S . = s must  be  the  extension  of  one  of  the  best 
t + 1 

sequences  S found  at  time  t with  S being  one  of  the  predecessors 

(Iff)  f 

of  s (i.e.  P(s)l  In  this  error  sequence  decoding  case  each  state  s 

may  have  a number  of  predecessors.  This  condition  gives  rise  to 

the  more  complex  syndrome  former  state  and  trellis  diagrams  as 

noted  in  Schalkwijk  and  Vinck  f 13"],  Thus  in  a Viterbi  Error  Sequence 
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Algorithm  at  time  t + 1,  each  syndrome  former  state  s must  have 

corresponding  storage  for  only  the  best  sequence  to  state  s and  t He 

associated  joint  probability  Pr(S  . . ,Z  . ,)• 

The  above  adaptation  has  considered  the  entire  state  sequence 

S . . For  real  time  (i.e.  finite  delay  = A)  Viterbi  error  sequence 

\ A » * t 

decoding,  only  A elements  of  the  best  sequence  for  each  state  must 

be  stored  along  with  Pr(S  , . , Z,.  , 

h (l.t)’  (l.t  1) 

Adaptation  of  Tee's  Viterbi  Algorithm 

With  the  above  clarifications,  Lee's  Real  Time  Viterbi 

Decoding  Algorithm  can  lie  easily  modified  into  the  Real  Time 

Viterbi  Krror  Sequence  Decoding  Algorithm,  Following  the  notation 

of  Lee  f 11  ],  the  necessary  metrics  and  sequences  can  be  defined  as: 

h(s)  = Pr(Sj'j  i for  all  syndrome  former  states  where 

S;,  , , is  the  best  path  S,.  , . with  S , = s and  B.(sl  = first 

( 1 . t + 1 ) K ( 1 . t+1 ) t + 1 i 

-(11 

digit  of  state  S . in  the  path  S + Note  that  B.(s)  = o'  . 

* t + 1 -r  ( 1 , t + 1 1 i t-i 

in  the  example  case  of  a rate  1 /n  systematic  code. 

Thus  the  following  algorithm  can  be  given. 

Real  Time  Viterbi  Krror  Sequence  Decoding  Algorithm  (A  > yl 


Step  0;  Set  t = 1 , Set  h(s ) = Pr(S^  2)’  ^ (l  1 ^ = l’r  (s  (?•  j /£,  s 1 

for  all  states  having  0 as  a predecessor.  Set  h(s!  = 0 
otherwise.  Set  B (si  - 0 for  1 si  £ \ and  for  all  states  s. 

i 

Step  1 : If  t £ A go  to  Step  2.  Otherwise  set  the  error  estimate 
- ( 1 1 

e ~B  (si  where  s is  the  state  for  which  h(sl  is  a maximuni. 
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Step  2;  Increase  t by  1.  Replace  R ^ .(s)  for  i = 1,2 

A-l  for  each  state  s. 

Step  3;  For  every  s make  the  replacement  his)  ♦-  Pr < s /s  1 )Pr(Z^/s,  s')* 
h(  s ) where  s'  is  the  state  maximizing  the  quantity  on  the  right. 
Set  Rj(s)  = 1st  digit  of  s'  for  each  s.  Return  to  Step  1. 

It  has  not  been  difficult  in  principle  to  adapt  a finite  delay  Viterbi 
data  sequence  decoding  algorithm  to  the  decoding  of  error  sequences. 
In  practice,  implementation  of  a Viterbi  error  sequence  decoder 
would  be  more  complex  than  a conventional  Viterbi  data  sequence 
decoder.  Unlike  data  sequences,  which  can  be  assumed  equilikely, 
error  sequences  are  very  non-equiprobable . Thus  the  metrics 
involved  in  a Viterbi  approach  must  be  actual  probabilities  (or 
logarithms  of  probabilities)  not  merely  simple  functions  of  Hamming 
distances  between  received  and  candidate  transmitted  sequences. 
Appendix  A,  "Viterbi  Algorithm  for  MAP  Estimation,  " has  additional 
details  of  the  adaptation  of  the  Viterbi  Algorithm  necessary  with 
non -equilikely  inputs, 

3.  5 Relation  of  Syndrome  Decoding  to  MAP  Data  Hit  Decoding 
In  any  kind  of  MAP  decoding  the  goal  is  to  estimate  some 
quantity  A given  some  other  quantity  R by  finding  that  value  of  A 
denoted  by  A which  maximizes  P(A/R). 

Since 
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P(A/B) 


P(B ) 


P ( B / A 1 P ( A ) 
P(B ) 


(3.93) 


i 


one  can  either 

(1)  determine  a computable  form  of  the  expression  for  P(A/B) 

A 

directly  and  then  find  the  A which  causes  P(A/B)  to  be 
maximum;  or 

(2)  determine  a computable  form  of  the  expression  P(B  / A)P(A)  /P(B ) 
and  determine  A which  causes  PtB  / A)P(  A ) /Pf  B 1 to  be  maximum. 
With  either  approach  one  basic  assumption  is  that  the  quantities 

A and  B are  related  by  some,  usually  probabilistic,  relation  or 
function. 

Suppose  we  now  apply  the  above  to  the  problem  of  decoding 
convolutional  codes.  Here  the  fundamental  problem  is  given  a 
received  vector  R determine  the  best  estimate  of  a particular 
information  bit  1^.  If  we  desire  to  use  a MAP  estimate  then  1 3.93  1 
becomes : 


P(R/lt)P(It) 

P'VS'  = — Piui — 


and  one  tries  to  determine  that  value  of  I (call  it  I ( 1 which  causes 
either  side  of  ( 3.94  ) to  be  maximum.  For  example,  this  is  the 
approach  taken  by  Me  Adam  f 9 "]  in  deriving  the  MAP  estimate  of  1 
given  R by  expressing  the  right  hand  side  of  ( 3.94  ) in  terms  of 

the  encoder  state  sequences. 
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Usefulness  of  Syndromes 


Now  suppose  one  wants  to  investigate  the  estimation  of  I using 
a computed  syndrome  (Z).  One  is  tempted  to  write  (3.93  ) as 


P(It/Z)  = 


P(Z/lt)P(It) 


(3.95) 


and  try  to  determine  that  value  of  I which  maximizes  either  side  of 
( 3.95  ).  However  this  would  be  wrong  since  the  basic  assumption 
that  1^  is  somehow  related  to  Z is  not  true.  Since  the  Syndrome  Z 
is  only  related  to  the  error  bits  {e|'^}  and  is  independent  of  1^,  no 
matter  how  intricately  one  expands  ( 3.95  ) even  in  terms  of  averag- 
ing over  received  vectors  R,  the  final  result  is  equivalent  to 


P(I(/Z)  = P(It) 


(3.96) 


Therefore  we  cannot  estimate  I,  given  7,  only. 


Actually  trying  to  estimate  I given  Z_  is  the  Data  Processing 
Theorem  at  its  worst.  By  the  Data  Processing  Theorem,  we  can  do 
no  better  than  estimating  I based  on  P(I^/R)  since  R_  is  our  received 
"raw"  data.  Any  "data  processor"  (c.g.  syndrome  former)  operating 
on  the  "raw  data"  R_to  yield  some  output  (e.  g.  Z)  will  tend  to  cause 
some  information  loss.  In  our  syndrome  case,  this  loss  is  all  the 
information  regarding  1^.  Since  the  errors  (e^)  are  independent  of  the 
data  (I()  and  the  syndrome  former  output  Z^  has  no  information  regard- 
ing 1^,  Z still  has  all  the  information  regarding  the  errors  {o'}  . 


Consequently,  we  can  just  as  well  estimate  e using  7_  by  maximiz- 
ing P(e^/Z)  as  we  can  estimate  e(  using  II  by  maximizing  P(e^/K_). 

A 

If  we  obtain  an  estimate  e by  either  method  we  must  still  go  back 

A 

to  the  raw  data  II  to  estimate  the  desired  1^. 

Since  finite  delay  A and  a discrete  memoryless  AWCN  channel 

a (i  1 

have  been  assumed,  the  error  estimate  e can  be  subtracted  from 

t - A 


ie  appropriate  rj'^  to  yield 


Mi)  (i)  MU 

C = r - e 
t-A  t-A  t-A 

known  estimated 


f 3.071 


With  estimated  available,  one  can  compute  1(  deterministic- 

ally.  For  the  binary  systematic  rate  1/2  case  sketched  in  Figure 

A ( 1 ) A 

3.7  , this  is  especially  simple  since  C = 1^. 

In  the  binary  estimate  case  shown 


j _(D  (M  ID 
!t  = ct  = rt  ®et 


(3.08) 


A MAP  estimator  for  I would  use  the  rule: 


Rule  1 


decide 


V-0 

P(It  = 0/R)  < P(It  = l /R  ) 

decide 


(3.001 


If  this  rule  is  rewritten  using  the  fact  that  1 = rj  0 ej  and 


the  fact  that  r^  t R_.  one  obtains 
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AU6  77  J T KINDLE 

AFIT-CI-79-13T 


F/6  9/9 


MICROCOPY  RESOLUTION  TEST  CHARI 


decide 


Pfrj1  ]9  e|1)  = 0/R)  * P(r|1}  ©e|1)  = l/R^) 

decide 


(3.100) 


The  binary  (mod  2)  sums  can  be  split  into  the  following: 

decide 

ft=° 

Plrj’^O.el'U/R)  + P(r|1)  = l,e|1)  = l/R)  * P(r|1^  = 0,e|1)  = l /R_) 


decide 


(3.101) 


+ P(r|1)=l,e^1)=0/R ) . 


Consider  the  only  two  possible  situations:  (1)  r^  =0;  or 
^=1.  In  these  two  situations,  Rule  1 is  equivalent  to 


Rule  1 . a 


decide 


1 } = 0 : P(e^=0/R)  * P(eJ1)=l/R) 


decide 


(3.102) 


Rule  1 . b 


decide 


r[1)=l:  P(e[1)=l/R)  * P(e|1)=0/R_) 

decide 


(3.103) 


A 


Instead  of  the  above  rule,  one  could  employ  the  following  error- 


It  is  easily  seen  that  for  all  possible  cases,  Rule  1 (as  imple- 

a 

mented  by  either  Rules  l.a  or  I.b)  will  give  the  same  estimate  1^ 

as  will  Rule  2.  Thus  both  methods  of  estimating  1^  are  equally  valid 

A 

in  the  systematic  code  case  considered.  That  is,  selecting  I to  be 

A 

that  value  of  1^  which  maximizes  Pd^AR)  is  equivalent  to  setting  1^ 
equal  to  the  value  r^  © e^  where  ej/'  is  that  value  of  e|^'  which 
maximizes  P(e|^/R). 

Basically  the  practical  reason  for  trying  to  use  a syndrome 
based  decoder  instead  of  a MAP  data  bit  decoder  using  _R  is  that  the 
implementation  might  be  easier.  The  syndrome-based  estimator 
portion  estimates  errors  with  a single  syndrome  Z containing  infor- 
mation on  errors  only  followed  by  a mod  2 addition  to  obtain  ^ in 
the  systematic  case.  On  the  other  hand  the  MAP  Data  bit  estimator 
must  work  with  _R  which  contains  information  on  both  data  and  errors. 

The  following  chapter  will  pursue  MAP  error  bit  decoding  based 
on  syndromes  under  the  dual  restrictions  of  finite  decoding  delay  and 
finite  syndrome  vector  length. 
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Chapter  4 

SUBOPTIMAL  ERROR  ESTIMATION 

In  this  chapter  certain  suboptimal  error  estimation  techniques 
are  derived  which  may  be  more  easily  implemented  than  the  optimal 
techniques  of  Chapter  3.  Specifically,  ROMMAP  ans  SUll MAP,  two 
suboptimal  approximations  to  the  optimal  realtime  (fini'e  delay) 

MAP  error  bit  estimator  of  Section  1.  3 will  be  considered.  Both  use 
a precomputed  look-up  table  of  error  bit  estimates  addressed  by 
syndrome  vector  values.  It  is  felt  that  such  a finite  delay  estimation 
problem  best  represents  the  case  to  be  encountered  in  practice.  In 
the  following,  the  restrictions  on  the  ROMMAP/SUBM AP  technique 
will  be : 

(1)  only  a specified  finite  delay  A >s  permitted  before  an  error  bit 
decision  must  be  made;  and 

(2)  only  a finite  length  syndrome  vector  Vj_  (or  received  vector  ft) 
is  available  to  the  decoder. 

The  ROMMAP  technique  will  be  first  shown  to  be  a particular 
average  of  the  optimal  finite  delay  estimator.  Next  the  actual 
methods  of  computing  MAP  error  bit  estimates  given  either  R^  or  '/_ 
will  be  derived.  Lastly,  SUBMAP,  a computationally  convenient 
approximation  to  the  suboptimal  ROMMAP  approach  will  be 
discussed. 

qo 


4.  1 RQMMAP  Derived  from  Optimal  Finite  Delay  Error  Decoding 


First  consider  the  optimal  finite  delay  MAP  error  bit  decoding 
of  Section  3.  3,  and  compare  it  to  the  suboptimal  ROMMAP  error 
decoding  approach  mentioned  in  Section  2.4.2.  The  optimal  finite 
delay  decoding  attempts  to  make  a MAP  estimate  of  an  error  bit 
e^  ^ using  the  computed  syndrome  bits  up  to  the  current  time  t but 
allowing  only  a finite  storage  of  syndrome  bits  in  such  a decision. 
However  it  incorporates  the  influence  of  syndrome  bits  prior  to  t-L 
in  making  this  MAP  decision.  ROMMAP  decoding,  on  the  other 
hand,  essentially  ignores  the  syndrome  bits  prior  to  t-L.  In  other 
words,  compare 


maximum 
over  all 
values  of 

*t-A 


P(e, 


.«>'} 


versus 


maximum 
over  all 
values  of 

*t-A 


OPTIMAL 


ROMMAP  (4.  1) 


where  for  now  we  have  taken  the  delay  A from  the  current  time  t 
to  be  equal  to  the  storage  delay  length  L. 

Note  first  that  the  required  probability  for  the  ROMMAP  and  the 
optimal  can  (via  Bayes  rule)  be  written  as  follows. 

ROMMAP: 


P(ex 


P(e  /z  ) = 

' t-A  (t-A.t) 


•a’ V.ytV 


P(Ztt-A,t)> 


(4.  2) 
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OPTIMAL: 


p,V/*ii.«,’  ■ 


Ple.-a-yd.t)1 


P(Ct-A,Z(I,t-A-I),Z(t-A,t)) 

P(Z(l.t-A-D’Z(t-A,t)) 


P(Z(t-A.t)/et-,VZ(I.t-A-I))P'et-A-Z(l,t-A-l)) 
P(Z(t-A»  t)/z' 1 , t-A-1  >)P(Z(1 , t-A-1  )> 


(4.  3) 


Now  consider  P(e  , z ).  Since  affects  z and  v 

t-A  t-/\  t - ^ 

subsequent  syndrome  bits  z z , is  statistically 

t-A+t  t-A-lv  t-A 

independent  of  z (the  past  syndromes).  Therefore, 

(* , t-A-t ) 

= P,'t-4,P"t(l,t-4-l|)  <4  4> 

and  the  Optimal  case  probability  of  interest  can  be  expressed  as: 
p(e  /.  , P(,<t-4.t/*t-rf*U.t-4-l)IP(*t-4IP,,(l.t.4-Ht 

Cancelling  equal  terms  of  numerator  and  denominator  in  the 
optimal  case  yields  the  following. 

OPTIMAL: 

pi.  h.  i = pi.c.‘-.^p!!ii--a. ' j1  ■ (4  6) 


ROMMAP: 


’•'Wlt-M)’  ' 


P(c.-4IP(A-A.t)/et-4' 


(t-A,  t ) 


(4.7) 
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Note  that  (4.6  ) and  ( 4.7)  are  similar  in  form.  The  basic 


difference  is  that  the  optimal  quantities  are  conditioned  on  the  past 

syndrome  bits  z . which  are  not  available  to  the  ROMMAP 

(* . t-A-1 ) 

approach. 

Equations  ( 4.6)  and  (4.7  ) can  simply  be  rearranged  to  give  the 
following  forms. 

OPTIMAL: 

,) 


pie  . . ^VvA'.n-n' 


't-A'  (1 . t)’ 


(4.  8) 


ROMMAP: 


P(e  /z  , _ p(tW!(t^.t)) 

( t-A7  (t-A,  t)'  P(Z/ 


(4.  9) 


(t-A , t)' 

Now  by  taking  the  formal  average  of  the  Optimal  Eq.  (4.8  ) or  Eq. 

(4.3)  over  z , one  obtains  the  ROMMAP  equation  (4.9).  That 

(t, t- A~t ) 


is 


E-E 

Z,  z 

1 •••  t-A- 1 


P(et-A/Z(l,t))  = 


P(et-A>8(i.t-.virVu)) 

P(z(l,t-A-l)’Z(t-A,t)) 


Average  of  OPTIMAL 


Z1  ’ ' • Zt-A-1 

pivii-m)1 

P|w>> 


P,VS/z(t-4.tl’  R°MMA!'  (4.10) 


Equation  (4.10)  shows  that  the  ROMMAP  technique  is  actually 
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just  what  unc  would  obtain  from  the  optimal  approach  if  one  was 
required  to  average  over  all  "past"  (i.e.  times  less  than  t-A) 
syndrome  bits. 

4.2  MAP  Estimate  of  e^  ^ given  Z 
1 -A- “ 

In  the  ROMMAP  approach  to  syndrome  decoding  of  the  example 
K-5  rate  1/2  systematic  code,  an  estimate  ej  ^ is  obtained  based  on 
the  computed  finite  syndrome  vector  Z_.  It  is  assumed  that  sueess- 
sive  syndrome  bits  {z|^j  are  computed  and  stored  in  associated 
shift  registers  of  length  L bits.  The  syndrome  vector  available  in 
this  example  is  thus: 


Z 


z(2)  ) 

’ t-L 


This  vector  would  appear  in  storage  elements  as  follows. 


z(2) 

t-L 

z(2) 

t-L+1 

• • • 

z,Z) 

t 

^ — L stages  of  storage  ► 

/j  vector 

It  is  useful  to  write  out  part  of  the  syndrome  equation  set  for  this 
example  using  the  convention  of  Eq.  (2.  18)  of  Section  2.  3. 


e(2> 


A r(2)  (D 

£ Gi,hVh 

h=0 


t-L  si  st,  v =4 


(4. 11) 


94 


This  is  done  below  in  Eq.  ( 4.  12  ) for  L > A and  can  be  used  to 


gauge  the  complexity  of  a particular  choice  of  L and  the  effect  of 

feedback  on  this  same  set  of  equations.  Shown  is  the  particular 

set  for  decoding  delay  A = 5 (i.e.  estimate  error  bit  ef^).  The 

t-  5 

dotted  line  designates  the  upper  "triangle"  of  error  bits  which  would 
have  been  previously  estimated  and  fed  back  if  error  estimate 
feedback  were  employed. 

(2)  (2)  (1)  (2)  (1)  (2)  (1)  _(2)  (1)  J2)(l) 

/t-L  Ct-L  G0  et-L  G1  et-L-l  C2  et-L-2+G3  t-L-3+G4  t-I^4 

v (2)  +r(2)(l)  +r(2)  (1)  (2)  (1)  J[2)(l)  (421(1) 

Zt-L+1"  ^-L+1+G0  et-L+l+C?l  et-L+G2  et-L-l+°3  et-L-2+C?4  el-L-3 


(2)  (2)  (1)  ' (2)  (1)  (2)  (11  (2)  (1)  (2)  (1) 

= et-5+C0  et-5+uC_l_  ^t-6+G2  et-74G3  et-8  + C4  Ct-9 

_ (2)  (2)  (1)  (2)  (1)~ ' _(2>  (1)  (2)  (1)  (2)  (1 ) 

- et -4  + G0  et-4  + G1  et-5  +l°2_  °_t-6  + C3  et-7  4 C4  et-8 

(2)  (2)  (1)  (21  (1)  . _(2)  (lP  (2)  (1)  (2)  (1) 

et-3  4 G0  et-3  4 G1  et-4+G2  et-5j_C3  et-64G4  et-7 

(2)  (2)  (1)  (2)  (1)  (21(1)  (2)  Tn"l  „(2)  (1) 

- °t -2  4 G0  “t-2  4 G1  °t -3  4 C2  et-4  4 G3  et-5  C_t-6_ 

(21  . _(2)  (1)  . _(2)  (1)  „(2)  (1 ) (2)  (1)  . _(2)  (1) 

“ t-1  G0  et-l  + G1  t-2  C2  t-3  4 G 3 t -4  4 C4  t-5 

_ (2)  . _(2)  (1)  _(2)  ( 1 1 (2)  (11  (21  (11  (21  (1) 
t G0  t 4 G1  t-1  4 °2  t-2  C3  t-3  4 G4  t-4 


(4.  121 


Given  an  Z vector  we  want  to  find  the  maximum  a posteriori 


probability  estimate  of  e which  in  the  systematic  code  example 

t-A 

(11  * 

can  then  be  mod  2 added  to  r to  give  I , the  estimated  (t-A)th 

t-A  t-A 

information  bit. 
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' is  thus  set  equal  to  that  value  of  which  maximizes  the 

t-A  t -A 

following  probability: 

max  over  all  (J)  (1)  (2)  (2)1 

values  v of  \ Pr  (e  =v/Z)  = Pr(e  =v/(Z  , . • . , Z ) > (4.13) 

l t-A  t “A  t-L,  t J 

e 

t-A 

But  this  can  be  expressed  as; 


max 

over 

v 


I (1)  . 1 max  | Pr(Z/.{*’=»)Pr(e{*bv)  ) 

{pr(et  ^v/Z)}  = over  p^§j j 

V v — 7 

|p r (_Z  /ej 1 v )P  r ( e | 1 v ) | 


max 
ove  r 
v 


(4.  14) 


This  decision  rule  requires  the  computation  of  Pr(Zye^*^=v). 

In  the  binary  case  the  rule  simplifies  to  the  following: 

decide 

£)  = 1 

PrlZ/c^vlIPrle^ll  | Pr(Z /e^vOlPrfe^eOI 

decide 


(4.  15) 


decide 

e^=0 
t-A 


For  example,  if  the  left  side  of  Eq.  (4.15)  is  greater  than  the  right  side, 

a(1) 


then  decide  e'  - 1;  if  the  left  side  is  less  than  the  right  side,  then 
t ~ A 

Ml)  _ 

decide 

Since  no  assumption  has  been  made  of  an  orthogonal  code, 

Pr(Z/e  =1)  for  example  can  be  expanded  in  terms  of  each  of  the 
t-A 

conditional  probabilities  of  the  other  error  bits  and  averaged  over 
the  probability  of  occurrence  of  each  error  bit  value  for  the 
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memoryless  channel.  For  the  BSC. the  Pr^e^  = l)  = p and  l5r(e^=0)  = 
q = 1 -p  where  p is  the  BSC  crossover  probability.  That  is, the  first 
step  in  the  expansion  would  be: 

Pr(Z/.|'bl)  = Pr(Z/e^=l.e^+1=l)Pr<e^+1  = l) 

This  expansion  would  be  continued  for  both  sides  of  the  decision 
rule  until  all  conditional  probabilities  for  all  error  bits  appearing 
in  syndrome  bit  equation  set  for  the  specified  |Z|  were  considered. 
An  error  vector  can  be  defined  as  in  Section  2.3  as  follows. 


(2)  (1) 


(D  Jl) 


E =(et-L,et-L et-A-l'et-A+l 

• 2 ( L-t  1 ) — 1 error  bits  in  E 


(Note:  e^  is  not  included  in  E.  ) 
t - A — 


el2>  eH) 

’ * ' t 9 t 


(4.  17) 


The  Maximum  A posteriori  Error  Bit  Decision  Rule  then  is 


decide 

;<>>=! 

V w(E)  2(L+1)-1 

P P ~ C1 

t-A 

-w(E)  > q 

^ w(E)  2(L+H-l-w(E) 

L p _q 

all  E 

satisfying 
this  7.  for 

e(1)=l 

t-A 

decide 

all  E 

A (1 ) n 
e =0 

t-A 

satisfying 
this  Z for 

(1)  A (4.18) 

et-<T° 

For  example,  if  the  left  side  of  Eq.  (4.  18)  is  greater  than  the  right 


side,  decide  e - 1;  if  the  left  side  is  less  than  the  right  side,  de- 

A 

cide  e^  ^ - 0. 

t-A 

For  a given  E^/K^  (and  tlierefore  a given  BSC  crossover  proba- 
bility (Pr(e^=  1))  the  above  computation  can  be  carried  out  for  each 
pos  sible  _Z  vector  in  the  set  {_Z  } of  sy nd rome  vectors. 

The  resultant  decisions  ej^(Z)  can  be  stored  in  a ROM  addressed  by 

t-A  — 

its  associated  Z.  Also  stored  could  be  the  actual  decision  confidence 


Pj(_Z)  where 


p,(Z)^p  2-J  qw(E)q2(L+l)-l-w(E) 

all  E satisfying 
this  Z with  e!  ^ = 1 


(4.  19) 


Tins  storage  of  the  set  fZ}  each  with  its  e^'|Z)  is  theROMMAP- 

t-  A 

the  Read  Only  Memory  Map  from  a computed  syndrome  vector  Z to 

its  corresponding  Maximum  A posteriori  Error  Bit  Estimate  q ^(Z). 

f- A 

Such  a look-up  table  incorporated  in  a decoder  enables  error  correc- 
tion with  essentially  no  decoder  computations. 


4.  3 MAP  Estimate  of  e'  Given  R 

t- A — 


It  is  instructive  to  now  temporarily  examine  just  what  it  would 
take  to  make  a ROMMAP-like  (i.  e.  , a look-up  table)  error  estimator 
using  the  actual  received  vector  R instead  of  a computed  syndrome 
vector  _Z.  This  diversion  will  demonstrate  why  the  syndrome  based 
approach  of  Section  4.  2 is  so  attractive. 

In  basic  MAP  data  bit  decoding  of  a systematic  convolutional  code. 


in  our  standard  binary  systematic  convolutional  code  example. 

McAdam  [91  found  a forward  and  backward  recursion  method 

A 

for  determining  I given  the  total  received  vector.  It  might  be 
useful  to  see  what  a ROMMAP  approach  to  finding  the  most  likely  I 
given  a finite  segment  R would  give  for  a binary  rate  1/n  systematic 
code  when  equilikely  data  inputs  are  assumed.  That  is, assume  a 
finite  decoding  delay  A and  storage  for  a finite  segment  R of  the 


received  vector. 


Recall  that  I = r^  ^-e*  ' = r^  © e^  ^ (over  GF(2)).  Then 


■i 


* _ that  I 

t maximizing 


t..  jpr(I  /R)\=  rf^  - that  {prfe^’/R)} 

mizing  ( t — ) t . t . . I t — ) 


maximizing 


(1) 


(4.  23) 


Since  in  the  systematic  code  case  r^  is  known,  finding  the 
maximum  a posteriori  estimate  of  1^.  given  II  can  be  accomplished 
by  finding  the  maximum  a posteriori  estimate  of  e|  ^ given  R_  and 

, , ..  "(1)  , U) 

subtracting  e^  from  r 

Now  concentrate  on  finding  that  value  of  e^  ' which  maximizes 

(1) 


Pr(e'  /R).  However, 


Pr(e|1  '=v/R_) 


Pr(Il/e|^=v)Pr(e^  ^=v) 
Pr(R) 


(4.  24) 


where  Pr  (e|  ^ = l)  = pandPr(e|  ^=0)  = l-p  = qfortheBSC.  R_is 
given  as  received  and  Pr(R_/e|  ^)  must  be  computed.  This  can  be 
done  using  the  same  approach  as  for  the  case  of  Pr(Z_/e|  ^) 
discussed  in  the  ROMMAP  determination  by  noting  that 


H)  = 0)+iji  l£j£n 

t t t J 


(4.  25) 


where 


.(j)  _ 


t 

k 


* v 

£ £ 

Z=l  n = 0 


GihXt-h 


if  1 sj  s k 

if  k < j £ n . 


(4.  26) 


Or, in  the  specific  case  of  k=l  (only  one  info  bit  per  block),  v=4,  rate 
1/2  systematic  code  of  the  example  this  becomes: 


(!)  r(M  . , U) 

r.  = C.  + e.  = I.  + e. 

1 11  xi 


100 


(4.27) 


(2)  (2)  (2) 

r.  = C.  + e. 

i i i 


v 

E 

h=0 


.(2)  T a (2)  „ T • * 

j. . I.  , + et  , t-Lsi  st  . 
In  i-h 


If  we  have  L bits  of  storage  for  the  (j)  r.,  1 £ j *;2,  received 
bit  streams,  our  available  R is: 

L.  stages  of  storage  ( 1 ) 


h 


r0! 

t-L 

t -L+l 

• • • 

r(1! 

t-1 

(2) 

rt-L 

r(2) 
t-E  + 1 

• • • 

12) 

rt-l 

t;1  -4- 


(2) 


.(2) 


To  show  a comparison  between  the  set  of  syndrome  equations  of 
the  ROMMAP  approach  (Eq.  4.  12)  with  the  corresponding  set  of 
equations  for  estimating  error  bits  from  an  R^  vector,  tie  comparable 
received  bit  equations  for  the  example  rate  1/2  code  is  given  below 
(Eq.  ( 4.  28  ) for  arbitrary  L. 


(D  (1)  , T 

rt-L  = et-L  + It-L 


(2)  _ (2)  (2)  (2)  (2) 

rt-L-et-L  + C0  It-L+°1  t - L - 1 + G*  1 


2 t-L-2 


+r(2)r  +C&L 
C'3  \-L- 3+G4  t-L-4 


Jl)  (1)  , T 

t-5  t-5  Tt-5 


■ (2), 


r.-5  * °t-5  ' Go"'.-5  * Cl\.b  4 4\-7  4 GP..8  > Ci"'It.9 


«-'(2l. 


.(2), 


(2)  (2) 

« = c 

t-5  t-5 

(!)  (1)  . . 
rt-4  “e,-4  +It-4 

r(2)  - <2>  ,r(2>r  4 r(2)i  , rlZ)i  l ^Z\  . r(2,r 

t-4  “ t-4  G0  t-4  G1  !t-5+G2  It-6+G3  It-7  + G4  *t-8 


(R  equations  continued  on  next  page) 
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(D  (1)  J , 

rt-3  =Ct-3  +It-3 

J2)  _ <Z)  . r(2)T  , r<2\  , r<2\  +r(2)T  +r(2)T 

t-3  t-3+G0  1 1 - 3 G 1 rt-4  + G2  It-5+G3  *t-6  + °4  *1-7 

(!)  (H  , T 

rt-2  Ct-2  4 *t-2 

(2)  (2)  (2)  (Z)  _(2)  x„(2)  . _(2) 

t-2  t-2  + G0  rt-2  G1  !t-3  4 G2  *t-4  G3  Xt  - 5 C4  *t-6 

(1)  (1)  , . 

rt  1 “Vi  Mt-1 

■•!2!  = <•,'-!  4 O<0\,  4 Ci2\-Z  4 G22\-3  4 G32\-4  4 Ci2\-5 

(2)  _ (2)  . n(2\  .-,(2)  , _(2) 
rt  t 4 G0  Tt  + G1  !t-l  G2  *t-2  G3  1 1 - 3 G4  !t  4 

(4.  28) 

A total  of  n(L  1 1 ) bits  arc  rece  ived  and  held  in  where  n=Z  in  t lii  s 

particular  example,  As  in  the  ROMMAP  computations  we  can  expand 

Pr(R  /e^  ^ ) in  terms  of  the  conditional  probabilities  for  all  available 
— t-A 

error  and  information  bits.  That  is  we  want  that  value  of  e'  which 
t - A 

maximizes  equation  (4.24).  But  this  is  equivalent  to  finding  that 
^ value  which  produces  the 

t-A 


max  over 
all  values 
(1) 


|pr(R/e[1))Pr(e(n  ) 
i — t-A  t-A 


(4.  29) 


This  value  can  be  compiled  from  equation  (4.  24)  by  expanding 

Pr(R  / e ^ ^ ) repeatedly  until  all  conditional  probabilities  for  both  error 
— t-A 

and  information  bits  have  been  incorporated.  The  first  step  would  be: 
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Fr(R/e#(I,=v)  = Pr(R/e[' U.x  =l)Pr(x  =1) 

— t-A  — t-A  k k 

+ Pr(R_/e|^=v,  xk=0)Pr(xk=0)  (4.30) 

where  x^  is  some  particular  info  o_r  error  bit.  At  each  step  in  the 
expansion  the  new  bit  introduced  (x  ) takes  on  another  identity  - 

K 

either  some  information  bit  (e.g.  I ) or  some  error  bit 

t - L4 k 

(e.g.  e^  ).  When  working  with  14  it  is  here  that  the  presence 

t — Li  'f  K 

of  information  bits  must  be  accounted  for  in  the  computation;  while 
in  the  syndrome  ROMMAP  case,  all  dependence  upon  information 
bits  was  removed  when  syndrome  bits  were  computed. 


Using  the  definitions. 


Error  vector  = K = 

r (2)  (1) 

et  - L’  Pt-L’  * ' 

. e"1  e(11 

e(2'  e(1)l 
” t ’ t 1 

(4.  31) 

and 

’ t - a-  r t - At  i ’ 

Information  vector 

= J--r'.-L-c- 

..•V 

(4.  32) 

provides  a notationally  simpler  form  for  the  expansion.  Note  tha1 

c[  ^ is  not  included  as  a component  of  the  E vector.  Also  if  x,  is  an 
t-A  — k 

error  bit  then  Pr(x  =1 ) = p and  Pr(x  =0)  = 1 -p  while  if  x is  an 
information  bit  then  Pr(x  =1)  = Pr(x  =0)  = \ (i.e.,  assuming  equi  - 

K K 

likely  source  inputs). 

The  quantity  Pr(R  /e^  ^ = 1 )Pr (e!  1 ) for  example  can  now  be 

— t-A  j-A 

written  as 
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Pr(U/^(  '^DPrte*  ^=1  ) = p^  ^pw(— )q2(  1 1-w(— ^(1=1  L^5~w(-)). 


all  _KI 
satisfying  this 

R with  e/  1 
1 -A 


(4.  33) 


But  with  equilikely  inputs  P(I=  1 ) = P(I=0)  = J and  since  ej  ^ is  1 or  0, 
the  new  form  of  the  decision  rule  is: 


Pr(R/e(1)=l)Pr(e(U  = l) 
— t-£  t-£ 


E 


w(  E)  2(  L+l ) - 1 -w(  K)  , . w(  I)  ,.(L+5)-w(I) 


all  E] 
satisfying  this 

R with  c\X)  =1 
— i-  A 


p — q 


-\b)  -\tr 


E 


decide 

> 

i'-V0 


all  El 

satisfying  this 

R with  ej  ^ = 1 
- t-A 


,w(  E)q  2(  L + 1 ) - 1 - w(  E)(  ^ } w(  I)(  ^ ( L+  5 ) -w(  I) 


PrfH/e!11 ' =0)Pr(e!I)  = 0)  . 
— J - A .1  - A 


(4.  34) 


For  equilikely  inputs  the  contribution  of  the  information  hits  to  each 
side  of  the  above  rule  is  the  same,  so  they  can  be  dropped  from  the 
rule  to  obtain 
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w(K)  2(1-1 1)-1 

p — q 


decide 


all  J01 

satisfying  this 

H with  e[  ^ \ = 1 
— t - A 


A 

-w(  E)  > > w(E 

- < q Zj  p - 

decide  all  El 


w(  E)  Z(LH)-l-w(E) 
— q — 


W l ) satisfying  this 

t_A  R with  cj1’  =0 

— t-  A 


(4.  35) 


Comparing  the  above  rule  Eq.  (4.35)  with  that  for  the  ROMMAP 

estimate  of  e^  ^ (Eq.  (4.  18)),  one  observes  that  the  quantities  being 
t- A 

summed  are  the  same  but  the  conditions  over  which  the  sum  is  taken 
are  not  identical.  In  the  ROMMAP  case,  the  sum  is  over  Error  Vec- 
tors J\  satisfying  the  current  _Z.  In  Eq.  (4.35)  above,  the  sum  is 
over  Error  and  Info  vectors  _EI  satisfying  the  current  _R.  Again  the 
problem  of  considering  t lie  information  bits  exists  when  one  estimates 
e^  ^ based  on  the  set  of  received  bits  R. 

t-A  ~ 

This  decision  rule  can  also  be  written  as 


decide 


Si”*! 


all  E 


,NEipW(E)q2(L41)-1-w(E) 


E)  > } ' w(E)  2 

“ < q4r'NE0P 


(Etl)-l-w(E) 


dec  ide 
e! 1 * =0 


(4.  3(i ) 


where  N...  ” number  of  I vectors  such  that  El  satisfy  current 
K1 

R with  e[  ^ - 1 
— t-  A 

and  NIV.  = number  of  I vectors  such  that  El  satisfy  current 
LU 

R with  e[  ^ = 0 . 
t - A 


At  each  step  in  the  computation  of  the  above  sum  over  all  E,  the 
following  applies; 

(1)  _R  is  known  from  the  received  bit  streams, 

(2)  K_  is  known  for  this  particular  step,  and 

(3)  we  need  to  compute  (he  nurnbe r of  I vectors  (out  of  a 

L+5 

total  of  2 possible  I_  vectors)  which  when  substituted 

along  with  this  particular  _E  into  the  "Ft  equations"  will 

satisfy  them  for  the  current  R . 

Since  It  is  given  and  is  known  for  this  particular  step,  Jt-E 

either  equals  a code  word  or  it  dot's  not.  It  is  assumed  here  that  L 

(the  number  of  stages  of  storage  for  each  received  bit  stream)  is 

greater  than  L where  L is  a constraint  length-determined 
min  min 

number  sufficient  to  guarantee  that  each  possible  input  bit  stream  is 

encoded  into  a unique  "codeword.  " With  this  assumption,  if  R-E 

is  not  a codeword  then  N_.  (or  as  the  case  may  be)  is  equal 

E 1 EO 

to  zero.  If  ’s  a codeword  then  it  is  a unique  codeword  and  only 

one  of  the  possible  I_  vectors  satisfies  the  R^  equations  for  this 
particular  K_. 

To  simplify  the  following  make  the  definition: 


A w(E  ) 2(L  + 1)-1  -w(E). 
f(EJ  = p — q — 


(4.  37) 


The  decision  rule  then  can  be  written  as: 


106 


all  El 
satisfying 
R with  eP 


decide  "1' 
f(E)  * 

decide  "O' 


(E)  ■ (4.  38) 


isfying  this 

ifith  eP  ) = 1 
t- A 


all  El 

satisfying  this 

R with  e =0 
— t-A 


Again  to  simplify  nota'ion  let  us  define  the  set  of  "codewords" 
(C  ] to  be  those  2(L+1  (-tuples  which  are  output  by  the  encoder  in 

K. 

rssponse  to  the  2LH^  possible  input  data  streams  under  consider- 
ation. Also,  for  those  which  are  considered  by  the  decision  rule 

sums  (i.e.  the  E whose  N . or  N equal  one)  we  can  apply 

El  E 0 


E = R ® C for  some  C €{C,  } . 

" ” ” K K K 


(4.39) 


The  decision  rule  then  can  be  written  as: 
decide 

e , "=  1 

23  l<  q 23  f< 

aHE=£  ©R  decide  all  E =C  OR 

for  all  Ck€(Ck}  «(1)  for  all  CR6  {Ck} 

with  ep  ' = 1 4 ^ withePpO 

t-A  t-A 


(4.  40) 


Eq.  ( 4.  40  ) above  then  equivalently  directs  the  decoder  to 
consider  all  possible  error  patterns  IE  which  could  have  mapped  any 
codeword  C,  into  the  actual  received  vector  R_  and  weigh  them 

K 

proportionately  by  their  probability  of  occurrence  f(IE)  to  make  the 

(1 ) 

maximum  a posteriori  decision  on  this  particular  error  bit  e^ 

Thus  the  additional  complexity  that  arises  by  attempting  to  estimate 


error  bits  e from  the  received  vector  R without  using  syndromes 
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has  been  demonstrated.  Consequently,  the  bulk  of  this  work  con- 


siders error  bit  estimation  on  syndromes  only. 


4.4  SUB  MAP  Approximation  to  ROMMAP 


In  Section  4.2  the  ROMMAP  decision  rule  for  estimating  the 
error  bit  for  K=5  rate  \ systematic  code  example  with  finite 

delay  A and  only  a finite  syndrome  vector  Z was  shown  to  be: 


ROMMAP  Decision  Rule 


decide 

t-j 


w(E)  2 ( L ( 
p — q 


l)-l-w(E)  > \ ’ 

" < 9 


w(E  1 2(L+1  M-w(E) 

p — q — 


all  f: 

satisfying  this 

7.  with  e(‘  ' = 1 
— t-A 


decide  all  E 

a/  J \ 

e'  -0  satisfying  this 

^ Zwithef'>=0 

— t-A 


(4.  41) 


This  ROMMAP  decision  rule  can  be  implemented  by  a read  only 
memory  addressed  by  _Z  and  containing  the  single  error  bit  estimate 
for  each  r7_  given  by  the  rule.  To  create  such  a look-up  table  would 
require  considering  all  error  patterns  [E'l  where  FP  = ( FI,  e^  ^)  (thus 
E'  contains  all  error  bits  including  the  one  to  be  estimated).  In  such 


a binary  case  this  means  considering  2 — separate  error  patterns 
where  |FP  | =the  number  of  error  bits  in  this  overall  error  vector. 
Such  an  exponential  growth  in  computational  complexity  is  a basic- 
drawback  to  any  look-up  table  decoding  approach. 

This  drawback  can  be  mitigated  somewhat  by  taking  advantage  of 


the  non-equilikely  nature  of  the  error  bit  values.  On  a practical 
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binary  symmetric  channel,  the  Pr(e|^=l ) « Pr(e|* *=0).  Thus  only 
"lightweight"  error  vectors  ((hose  having  few  l's)  have  a significant 
probability  of  occurring.  Therefore,  if  instead  of  considering  all 
possible  error  vectors  (hPl  as  dictated  by  the  ROMMAP  decision  rule, 
one  was  to  consider  only  a lightweight  subset  of  [E'l  , the  computa- 
tional complexity  of  creating  the  look-up  table  could  be  reduced.  The 
resultant  SUBMAP  look-up  table  would  not  be  significantly  different 
than  that  of  the  full  ROMMAP  since  the  neglected  contributions  to 
the  sums  duo  to  "heavy"  E_  ’n  *1.  41  would  be  extremely  small.  Thus 
the  decisions  found  for  either  table  would  be  almost  the  same. 

To  see  the  complexity  reduction  in  look-up  table  computation  , 
consider  the  K = 5 rate  \ code.  If  a syndrome  vector  of  LSYN  =15 

syndrome  bits  is  considered,  the  look-up  table  in  either  case  would 
, 1 5 , LSYN 

contain  2 =2  locations.  The  ROMMAP  computation  would 

require  considering  2 ' 1 - 2 ' **  = 1 7,  1 79 , 86%  2 1 0 err  or  vectors  E'. 

If  instead  one  only  includes  error  vectors  of  weight  5 or  less,  then 

34  34  34  i4 

the  SUBMAP  computation  requires  considering  (1  -4  ( ) 4 (~  1 + / ) + 

0 1 ' X 

34  34 

(4)4(5)  = 331,212  error  vectors  for  a complexity  reduction  by  a 
factor  of  approximately  51,  870. 

For  completeness  the  SUBMAP  computation  rule  for  the  previous 
rate  ' binary  example  may  be  written  as; 


SURMA  P Decision  Rule 


decide 
e(1)=l 

t-t  sr^ 

w(E)  2(L+l)-l-w(E)  > > w(E)  2( L+ 1 ) - 1- 

p — q — q P — q 

all  El  decide  satisty‘n8  this 

satisfying  this  _Z  such  that 

Z such  that  w(  E)  s 4 e =0  w(E)  ■&  5 

t-  a — 


E 


■w(  E) 


with  e 


(1) 

t-A 


(4.42) 


Due  to  the  rapid  exponential  growth  in  computation,  in  this  work 
the  full  ROMMAP  computation  was  carried  out  only  for  a K = 5 rate  A 
code  for  the  case  |_Z|  = 6 = LSYN  syndrome  bits.  All  subsequent 
work  for  longer  constraint  length  codes  or  more  syndrome  bits  con- 
centrated on  the  SURMAP  approsch.  It  is  shown  in  Chapter  5 that 
there  is  available  a very  tight  upper  bound  on  performance  using 
the  SURMAP  approach  which  enables  the  accurate  prediction  of  per- 
formance simultaneously  with  the  creation  of  the  SURMAP  look-up 
table. 


A specific  comment  is  in  order  on  the  relationship  between 

ROMMAP,  SURMAP,  and  conventional  feedback  decoder  approaches. 

The  conventional  feedback  decoder  as  described  by  Heller  [16]  uses 

as  the  error  bit  estimate  that  value  of  appearing  in  the  minimum 

t-  a 

weight  error  vector  that  could  have  caused  the  given  syndrome 
vector  Z.  This  means  that  the  many-to-one  mapping  of  {E]  vectors 


into  syndrome  vectors  f/~)  is  .approximated  by  a one-to-one  mapping 


using  only  one  of  the  lightest  weight  error  vectors  _E.  This  is  con- 
trasted with  the  ROMMAP  approach  in  which  the  entire  many-to-one 
mapping  is  considered  in  making  the  error  bit  estimate  and  with  the 
SUBMAP  approach  in  which  a significant  portion  of  the  many-to-one 
mapping  is  used. 

Thus  the  three  approaches  can  be  ranked  by  how  close  they 
approximate  the  true  finite  delay-finite  syndrome  MAP  estimate: 


Approach 

Mapping  Considered 

A 

e is : 

ROMMAP 

Entire 

MAP  estimate 

SUBMAP 

Subset  of  entire 

Approximate  MAP 

estimate 

C onventi  onal 

Single  E 

Grosser  approximation 

Feedback 

to  MAP  estimate 

Note  that  in  all  three  approaches  the  size  of  the  decoder  look-up 

\Z  I 

table  is  the  same  (2  — ' in  the  binary  case).  Thus  one  is  logically  led 
to  assume  that,  given  a specification  on  the  maximum  | Zj  (and 
consequently  the  maximum  size  of  the  look-up  table),  it  would  be 
advisable  to  use  look-up  table  entries  computed  by  considering  as 
large  a subset  of  [El  vectors  as  possible  thus  approaching  the 
ROMMAP  or  true  Finite  Delay-Finite  Syndrome  MAP  error  bit 
estimate. 

Chapter  5 demonstrates  that  this  is  indeed  the  case  since  both 
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computed  performance  and  simulation  results  show  that  the  SUBMAP 
decoders  outperform  conventional  feedback  (i.e.  minimum  weight  E ) 
decoders.  In  many  cases  a SUBMAP  decoder  even  without  feedback 
will  perform  better  than  a conventional  feedback  decoder. 
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Chapter  5 


PERFORMANCE  PREDICTION  AND  SIMULATION  RESULTS 

In  Chapter  3 several  theoretically  optimal  error  bit  decoding 
approaches  were  derived.  Chapter  4 took  one  of  these,  optimal  finite 
delay  error  bit  decoding,  and  exhibited  two  suboptimal  approximations, 
the  ROMMAP  and  SUBMAP  decoders.  In  this  chapter  decoder 
performance  prediction  is  analyzed  for  these  two  decoders  when 
implemented  for  typical  codes.  It  is  shown  that  an  actual  probability 
of  bit  error  can  be  computed  for  the  ROMMAP  decoder  while  a tight 
upper  bound  on  performance  of  the  SUBMAP  decoder  can  be  obtained 
as  the  look-up  table  is  created.  A lower  bound  based  on  an  idealized 
first  error  probability  is  developed  and  shown  to  give  a lower  limit 
to  continued  performance  improvement  for  a given  code  and  decoding 
delay.  Also  the  performance  improvement  possible  both  by  changing 
to  a longer  constraint  length  code  and  by  increasing  |Z_|  for  a given 
code  is  displayed.  Lastly,  Monte  Carlo  SUBMAP  simulation  results 
for  two  codes  using  various  lz|  values  with  and  without  feedback 
are  presented  and  compared  with  conventional  feedback  decoder 
performance . 

5.  1 ROMMAP  Creation  and  Predicted  Performance 

In  this  section  the  actual  computation  of  a ROMMAP  look-up 
table  to  implement  a binary  MAP  finite  delay-finite  syndrome  error 
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decoding  rule  is  qualitatively  described  along  with  the  manner  of 

computing  the  actual  probability  of  bit  error  for  the  ROMMAP 

decoder.  This  look-up  table  is  addressed  by  the  current  syndrome 

vector  value  Defining  LSYN  = |_Z  ) = the  number  of  syndrome  bits 

in  the  syndrome  vector  means  that  the  size  of  the  ROMMAP  table 
, .LSYN 

must  be  2 . Each  addressed  storage  location  in  the  table  must 

contain  either  a 0 or  1 value  corresponding  to  the  MAP  error  bit 

A (1 ) 

estimate  e determined  by  the  rule.  This  value  will  sometimes  be 
t-A 

a(1) 

denoted  as  e^  ^ (_Z)  in  the  following  explanation  for  a binary  rate  j 
systematic  code. 

5.1.1  ROMMAP  Creation 

It  should  be  specifically  noted  that  the  two  entities  of  interest  to 
our  decoder  for  a rate  \ systematic  code  are: 

(1 ) = (z|  \ zf  | , . . . , z|  | (provided  in  real  time  by  the 
syndrome  former  and  syndrome  bit  shift  register);  and 

(2)  e|  ^ (the  error  bit  to  be  estimated  by  reference  to  the 
ROMMAP  look-up  table). 

Here  L is  the  "syndrome  bit  delay  parameter"  indicating  that  L+l 

consecutive  syndrome  bits  comprise  the  syndrome  vector  Z.  It  is 

(2) 

always  assumed  that  Z^  is  the  latest  syndrome  bit  output  by  the 
syndrome  former.  Also  A is  the  "error  bit  delay  parameter" 
indicating  that  A+l  noisy  information  bits  rj*  j = t,  t-1,  , . . , t-A  , are 


delayed  and  available  to  be  corrected.  This  means  that  at  time  t, 
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r 

■ 


i 


when  ^ and  become  available  to  the  decoder,  the  ROMMAP 

t t 

A(1  ) *(1  ) 

decoder  will  produce  e by  "looking  up"  the  value  e ( Z 1 in  the 

t - A t - A 

ROMMAP  table  and  mod  2 add  it  to  rf*'  to  give  the  information  bit 

t -A 

estimate 

? (M 

't-6  = rt  - i + et-  i — 

Consequently  the  "decoding  delay"  is  A. 

The  distinction  between  A and  L is  worth  making  clear  since, 
although  syndrome  decoding  schemes  such  as  conventional  feedback 
decoding  usually  implement  L =■  A,  there  is  no  inherent  reason  why 
the  decoding  delay  A must  equal  the  syndrome  bit  delay  L.  In  the 
work  that  follows  Li  1, 

For  convenience,  the  actual  implementations  discussed  in  this 
and  later  sections  were  restricted  to  binary  systematic  codes.  For 
such  codes  the  total  ROMMAP  table  approach  may  be  mechanized 
in  a hardwired  device  which  only  has  to  determine  if  the  current 
syndrome  vector  Z belongs  to  the  subset  of  syndromes  such  that 

(Z)  = 1.  That  is,  the  contents  of  a ROMMAP  table  actually 

t - A — 

map  the  2^^^  syndrome  vectors  into  two  points,  (0,  1).  A study 
of  the  redundancy  in  the  syndr ome s -t o -decisi on  mapping  was  not 
pursued  in  the  work  but  would  seem  to  be  a possible  means  of 
speeding  decisions  when  compared  to  a slow  look-up  table  search. 

For  later  reference,  the  ROMMAP  decision  rule  for  a systematic 


rate  \ code  is: 
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decide 
t-A 

. LEN-w(E)  > 


Zw(E)  LEN-w(E)  > v y"  w©,  . 

p — (i-p)  — < (i-p)  2_,  p ~(1-p) 


w<EL  LEN-w(K) 


all  E 

satis  fying  this 

Z with  e * ' »1 
— t-A 


decide 

MD  n 

e =0 
t-A 


all  E 

satisfying  this 

t-( 1 >=0 


Z with 


t-A 


(5.2) 


where:  E is  the  vector  of  all  error  bits  entering  into  the  set  of 

LSYN  syndrome  equations  other  than  the  hit  ej  ^ to  he  estimated; 

LEN  is  tin-  length  of  E ; /.  is  the  current  syndrome  vector  of  length 

I'll  = LSYN;  and  p is  the  BSC  crossover  probability.  Note  that 

Eq.  (5.2)  is  merely  the  decision  rule  resulting  from  considering 

only  the  numerator  of  the  ROMMAl’  probability  in  Eq.  4.14  and 

specializing  to  a rate  binary  case. 

For  the  relatively  short  constraint  length  binary  examples  of 

ROMMAP  and  SUBMAP  decoders  considered  here,  the  first  step 

in  implementation  is  the  creation  of  the  look-up  table,  i.e.  the 

mapping  from  syndrome  vectors  /,  to  a part  icular  error  hit  estimate 

e|  ^ . In  the  ROMMAP  case  this  mapping  is  done  by  deterministically 
LEN 

generating  all  2 error  patterns,  substituting  each  into  the  right 

hand  side  of  the  appropriate  set  of  syndrome  equations  with  the  error 

LEN 

bit  to  be  estimated  set  to  0 and  then  to  1.  For  each  of  the  2 
error  patterns  this  gives  two  syndromes  (equivalent  to  each  of  the 


lib 


overall  error  patterns  mapping  to  one  syndrome).  A data 


r 


2'-™" 

structure  consisting  of  a double  vector  of  probabilities  indexed  by 

the  2^  ^ ^ possible  syndrome  values  was  used  to  accumulate 

contributions  to  P(/./e!  ^=1)  and  P(Z/o^  ^=0)  as  all  error  patterns 
— t - A — t-A 

were  processed.  Once  all  error  patterns  had  been  processed,  the 
data  structure  entries  are  multiplied  by  p and  1 -p  respectively  to 
yield  the  terms  of  the  decision  rule  (5.2),  for  each  Z.  By  cycling 
through  all  Z locations  in  the  data  structure  the  MAP  error  bit 

A ( 1 ) 

decision  0 ( Z > at  each  Z can  be  made  based  on  these  terms  and 

t - A 

recorded  in  a third  level  of  the  data  structure  to  give  a triple  vector. 
This  third  level  is  the  ROM MAP  look-up  table  as  shown  in  Figure  1. 
5.1.2  Signal  to  Noise  Ratio 

One  should  note  here  that  since  the  contributions  to  the 
l’(e|  ^ 1 / Z ) and  P(e|  |~0/Z ) numerators  are  based  on  the  crossover 

probability  p of  the  binary  symmetric  channel,  it  is  necessary  to 
consider  how  this  p is  related  to  the  signal  to  noise  (SNR  ) level  of  the 
received  signal.  For  a particular  F vector  of  weight  w(F)  for  example, 
the  contribution  would  be 


w(F),  LFN  t 1 -w(F ) it  , , ill 

p — ( 1 - p ) — for  the  syndrome  when  0^  - 0 and  (5- 'a) 


w(E)+l  .LKN-w(F)  . (1)  - 

p — ( 1 -p)  — for  the  syndrome  when  e^  - 1 


(5-  M») 


The  crossover  probability  p can  be  related  back  to  a signal  to 

noise  ratio  assuming  additive  white  Gaussian  noise  and  a particular 

1 1 7 


Add  ress 


1st  level 


2nd  level 


3rd  level 
(decision) 


Figure  5. 


1.  Triple  Level  Data  Structure  tor  ROMMA1 
Creation. 


modulation.  Thus  any  ROMMAP  look-up  table  has  associated  with  it 
a "SNR  design  value"  corresponding  to  the  p used  in  the  probability 
contribution  computations. 

In  the  case  of  180  degree  binary  phase  shift  keying  'BPSK) 
modulation  assumed  throughout  (his  work,  one  can  easily  relate 
the  BSC  crossover  probability  p to  SNR  level.  The  additive  white 
Gaussian  noise  causes  the  received  channel  symbol  level  v to  have 
the  conditional  distributions  shown  in  Figure  5.2  assuming  proper 
receiver  gain  adjustment  to  provide  unit  variance  as  noted  by 
McAdam  [ 9 1 . Note  that  in  Fig.  5.2  the  mean  of  the  conditional 
dist  ributi  ons  is  i^E  /N^  and  the  cross -hatched  area  can  be  used  to 
compute  the  BSC  crossover  probability  p. 

Thus 


■ r 


v=+® 


V = +0D 


P(v/0)dv 


"v=0 


f 

v=0 


— - 2 


— exn"^(V+'/2Es/N^ 
\/2tt  exp 


s'^0’  dv. 

(5.4) 


With  x = v + ./ 2E  /N  ' one  has 
s 0 


■fCG 

■J 


x=+  /2E  / 'NA 
s 0 


and  with  u = x/v/7' one  obtains 

2 


1 -lx2 

exp  ‘ dx 


•}  CO 


j p w 2 u 

= Tvf  I exP  U du  = >;  (-T.  [ e”U  du=4  erfc 

v17  nr  'em  J ^ s 


u=y E /Nrt 
v s 0 


“='/e7n7 


s 0 
(5.5) 


whe  re 


^ OB  L 

2 p -t 

crfc(x)  = - j= y J exp 


dt. 


This  can  also  be  expressed  as 

2 X 

P = i[l  -erf(Eg/N0)l  where  erf(x)  = — ,J  exp  dt 


(5.  6) 


as  noted  in  Wu  [ 1 5]  . In  the  above  it  should  be  emphasized  that 


and 


E = average  signal  energy  per  transmitted  bit  (symbol  energy) 


Nq  = noise  power  density. 


If  one  desires  p for  BPSK  to  be  expressed  in  terms  of  the  more 
conventional  E,  , the  uncoded  signal  energy  per  bit  for  a rate  R 
code,  then,  using  Eg/N0  = (E^NqIR.  the  BSC  crossover  probability 
become  s : 


p = l[l-erf(EbR/N0)l 


(5.7) 


where  R - the  code  rate. 

Since  all  codes  actually  implemented  and  compared  in  this  work 

were  rate  R = both  predicted  and  simulated  data  for  probability  of 

bit  error  are  plotted  in  terms  of  E /Nrt  in  db,  the  transmitted  symbo 
r s 0 

signal  to  noise  ratio.  This  is  consistent  with  the  approach  of  other 

authors  (e.g.  Heller  [ , Schalkwijk  and  Vinck  [131)  who  tend  to 

plot  "P  versus  P However  some  caution  must  be  used  in 

r out  in 

comparing  results  of  other  work  (e.g.  Heller  and  Jacobs  [ 1 ?1  ) 

where  P (equivalent  to  P herein)  is  plotted  in  terms  of  E /N  . 

out  e ii  n 
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5.  1.  3 ROMMAP  Probability  of  Bit  Error 


Once  the  ROMMAP  look-up  table  has  been  computed,  sufficient 
information  is  available  in  the  aforementioned  data  structure  to 
actually  compute  a probability  of  error  for  the  error  decision.  The 
set  of  addresses  ( _Z  } of  the  ROMMAP  table  can  be  divided  into  two 
disjoint  subsets: 


Z = [the  set  of  all  syndrome  vectors  for  which  look-up 

table  entries  e (Z)=0l  (5.8) 

t-A 

and 

Z.  = (the  set  of  all  syndrome  vectors  for  which  look-up 

1 A 

table  entries  e (Z)  = ll 

t-  A — 

Thus  Z = Zg  UZj  and  Z^  P Z^  = 0.  The  ROMMAP  decoder  operates 
by  deciding: 


and 


et-A  = ° *f  — -*Z0 

Va=  1 * 


'5.9) 


To  determine  the  probability  of  making  a wrong  error  bit  decision 
it  is  convenient  to  define: 


= [probability  of  making  a wrong  error  bit  decision  when  e =0j 

t-  A 


■ E p£/vs=°> 

z«zi 

and 


(5. 10) 
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P . = [probability  of  making  a wrong  t»rror  bit  decision  when  e = 11 

e, I t-£ 


■ E 


(5. 11) 


£‘zo 


Averaging  over  both  possible  values  of  e^  gives  the  average 


error  probability  P^: 


P = P(e  =0)P  4 P(e  =1)P  , 

e t-A  e, 0 t-A  e, 1 


(5.12) 


0) 


The  quantities  P(Z/e  =0)  and  P(Z/e  =1)  appearing  in  (5.  ll 
— t-A  t-A 

and  (5.11)  are  the  same  as  the  quantities  computed  in  creating  the 
ROMMAP  look-up  table.  Thus  (5.  10)  and  (5.  11)  can  be  evaluated  for 
the  particular  code  and  particular  LSYN  value  by  summing  over 

A A 

those  data  structure  addresses  Z for  which  e (Z)=l  and  e (Z)=0 

— t-A — t-A — 

respectively.  Which  Z is  in  which  subset  is  already  tlagged  by  the 
0,  1 entries  at  each  address  '7.  This  implies  that,  for  a particular 
systematic  binary  code,  the  computation  necessary  for  the  creation 
of  a ROMMAP  table  also  provides  a theoretical  value  for  probability 
of  decoding  error  P^  at  the  design  SNR.  For  example  Figure  5.4 
shows  ROMMAP  P as  computed  for  a K=5  code  versus  design  SNR 
(lower  curve). 

Since  the  actual  ROMMAP  performance  can  be  calculated,  the 
use  of  any  Chernoff  or  Simple  Bound  approach  as  used  in  Gallagher 
[l8*l  for  performance  prediction  is  not  warranted.  Indeed  when  one 
pursues  either  approach  it  is  found  that  the  same  quantities 
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P(Z/e  =0)  and  P(Z/e  =1)  are  needed  for  either  bound  evaluation 
— t-A  — t - A 

so  one  might  just  as  well  compute  directly  as  described  above. 

Thus  is  has  been  shown  that  if  a look-up  table  is  created  for  a 
ROMMAP  decoder,  an  actual  probability  of  bit  decoding  error  is 
readily  available.  However,  as  noted  in  Section  4.4,  the  number  of 
error  vectors  IC  that  must  be  considered  for  the  full  ROMMAP  grows 
exponentially  with  syndrome  vector  length,  LSYN.  Thus  one  is  led 
to  consider  the  SUBMAP  look-up  table  approach  and  a method  of 
predicting  or  bounding  P^  for  SUBMAP  decoding.  This  bound  is 
considered  in  the  following  section. 

5 . 2 SUBMAP  Creation  and  Performance 

Due  to  the  exponential  growth  in  computational  complexity  with 
syndrome  vector  length  for  the  ROMMAP  decoder,  it  is  feasible  to 
use  the  full  ROMMAP  approach  only  for  quite  short  constraint  lengths. 
Consequently  it  is  worthwhile  to  consider  creating  a SUBMAP  look- 
up table  decoder  as  discussed  in  Section  4.4.  This  will  be  done  in 
the  following  sections  along  with  consideration  of  performance 
prediction  and  the  relationship  to  conventional  feedback  decoding. 

5.2.1  SUBMAP  Creation 

In  computing  a SUBMAP  look-up  table,  the  same  three  level 
data  structure  described  above  for  the  ROMMAP  approach  is  used. 

It  is  loaded  with  probability  contributions  as  before  but  only  a 
lightweight  subset  of  all  possible  Ejvectors  's  considered.  Thus 
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contributions  from  the  subset  of  E1  vectors  have  been  entered,  the 


data  structure  entries  at  each  Z address  are  P 17. /e  ),  P'(7./e  1, 

— — t-A  — t-A 

and  D(Z)  = e^  \z).  The  is  used  to  differentiate  these  quantities 

— t-A  — 

from  the  corresponding  ROMMAP  quantities  since  they  will  be  wlv 
approximately  the  same.  The  SUBMAP  entries  will  be  less  than  or 

equal  to  the  ROMMAP  entries.  That  is  P(Z/e  ) = P (Z/e  1 ^ 6 

— t-A  — t-A 

where  6 is  the  difference  due  to  the  neglected  "heavyweight"  sectors 
If  the  maximum  weight  of  error  vectors  (MAXI  in  the  lightweight 
subset  is  increased  toward  LEN+1  the  SUBMAP  and  ROMMAP 
tables  will  become  identical.  Just  how  similar  the  performances 
of  the  two  approaches  will  be  is  explained  in  the  following  section. 

For  reference  purposes  the  three  level  SUB  MAP  data  structure  can 
be  visualized  as  depicted  in  Figure  5.  3. 

5.2.2  A Tight  Upper  Bound  On  SUBMAP  Performance 

Since  the  SUBMAP  approach  creates  a look-up  table  by  consider- 
ing only  a lightweight  subset  of  the  total  set  of  possible  error  vectors, 
the  effect  on  performance  of  the  excluded  error  vectors  must  be 
measured  or  bounded. 

Consider  expressing  the  average  error  probability  P for  the 
SUBMAP  case  as  follows: 
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Figure  5.3.  Triple -level  Data  Structure  for  SUB  MAP 
creation. 


I 
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P = ^ Plerror  /Ej)P(E|)  = P'err  or /E)P(I?) 

all  Is1  all  _E' vectors 

vectors  considered  in 

SUBMAP 


+ ^ P (error  /E|)P(E|)  (5.13) 

all  E vectors 
excluded  from 
SUBMAP 

In  the  second  summation  above,  the  quantity  P(error/F.')  S 1.  To 
upper  bound  P , set  Pferror/P)  = 1 only  in  the  second  term.  In 
other  words  assume  a decoding  error  is  made  with  probability  equal 
to  one  whenever  an  excluded  E'occurs.  This  yields  the  bound: 


Pe  < ^ P(error /E)P(j?)  + (l)P(J?)  (5.14) 

all  E_' vectors  all  E^' vector  s 

considered  in  excluded  from 

SUBMAP  SUB MAP 

Now  for  notational  convenience  define: 

(1)  D(Z)  = e (Z)  = the  SUBMAP  table  0,  1 decision  addressed  by  Z; 

— t-&  — 


(2)  ZfE1)  = the  syndrome  vector  which  corresponds  to  a particular 


error  vector  _E ' ; 


(3) 


P = P(error /E)P(E):  and  (4)  PgR.p 

all  E'vectors 
considered  in 
SUBMAP 


XI  (l)P(E'. 

all  E'vectors 
excluded  from 
SUB  MAP 

(5. 15) 


Thus 


(5.16) 


P £ P + P , . 
e skip 

Now  consider  P.  A decision  mistake  can  be  made  in  the  follow- 
ing two  ways  when  an  E]  vector  contained  in  the  subset  considered  in 
deriving  the  SUBMAP  table  is  present: 

(1)  the  error  vector  E'is  such  that  e =1  but  D(Z(E))  = 0;  or 

t-A  

(2)  the  error  vector  E'is  such  that  e =0  but  D(Z(E))  = 1. 

t-A 

In  case  '1)  the  contribution  to  the  sum  due  to  such  an  E'having 

e =1  is 
t-A 

Pferror/E^PfE')  = P(D(Z_(E^)=0)P(E|) 

= (1-D(Z(E'))P(E')  . (5.17) 

In  case  (2)  the  contribution  to  the  sum  due  to  such  an  E'having 

et  =0  is 
t-A 

P (error  /E^)P(E^)  = P(D(Z_(E'))  = 1 )P(E') 

= D(Z(E'))P(E_r)  (5.18) 

By  the  previously  described  method  of  loading  the  look-up  table 
data  structure: 

P*(Zy et_A=l ) = Yj  p(E)  and  P*(Z/et_=0)  = £ P(F) 

all  E1  in  SUB  MAP  all  E'  in  SUBMAP 

(with  et  =1)  mapping  (with  et_^=0  ) 

to  this  77_  mapping^  to  this  Z 

(5. 19) 
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Therefore, 

p = ^ (1  -D(/_))P  ■ (Z_/et  = 1)  + ^ D(^-)P  ' (— (5*20) 

all  Z all  Z 

and  P can  be  easily  calculated  from  the  three  level  SUBMAP  data 
structure  entries. 

Now  consider  P . . This  is  merely  the  probability  that  an 

skip 

error  vector  E'occurs  which  has  not  been  considered  in  creating  the 
SUBMAP.  Such  an  error  vector  must  have  weight  greater  than  MAX, 
the  maximum  weight  of  error  vectors  considered  in  the  SUBMAP. 

For  notational  convenience  let  LNFR  = LFN+1  = the  number  of  bits 
in  the  overall  error  vector  F*  (including  the  bit  to  be  estimated). 


Also  note  that  the  probability  of  having  present  an  error  vector 
weight  w of  length  LNFR  is 

P . (“E»)p'|..p|““'W  (5.211 

w V w / 

Since  all  E'vectors  having  more  than  MAX  ones  have  been 


excluded, 


skip 


£ 

all  F' 
excluded 
from  SUB  MAI 


LNFR 

P(J2)  = P. 

i=MAX  + l 


MAX 

£ P.  (5.22) 

i =0 


Thus  it  is  a straightforward  matter  to  compute  both  (5.20)  and 
(5.22)  to  yield  an  upper  bound  on  the  actual  P using  the  quantities 


12b 


already  available  from  the  SUBMAP  look-up  table  computation. 


This  bound  will  be  quite  tight  especially  at  higher  SNR  levels 

where  p is  small  thus  reducing  the  magnitude  of  P . Figure  5.4 

skip 

shows  the  tightness  of  this  upper  bound  as  MAX  is  increased  from  1 

to  5.  For  the  K=5,  LSYN  = 6 decoder  case  shown,  one  notices  that 

the  bound  is  looser  at  low  SNR  and  tighter  at  high  SNR.  Also  for 

MAXsd  = 4 there  is  little  difference  ( £0.1  db)  between  the 
min 

theoretical  ROMMAP  performance  and  the  upper  bound  on  SUBMAP 

performance.  However  since  the  SUBMAP  creation  takes  significantly 

less  computations  than  the  ROMMAP  approach,  it  represents  a much 

more  feasible  way  to  gauge  performance.  With  the  proper  choice 

of  a MAX  weight  considering  the  available  minimum  distance  d of 

min 

the  code,  one  can  closely  approach  ROMMAP  performance  with  a 
SUBMAP  decoder.  The  predicted  performance  shown  by  the  bounds 
in  Figure  5.4  will  be  confirmed  by  simulation  in  Section  5.5. 

5.2.3  Performance  Comparison  with  Conventional  Feedback 

The  SUBMAP  look-up  table  is  created  by  considering  a lightweight 
subset  of  FT  vectors  specified  by  MAX.  Included  in  this  subset  is  at 
least  one  mi ni mum  weight  11' vector  mapping  to  each  syndrome  _Z. 

Such  a single  E' vector  is  commonly  used  to  make  the  look-up  table 
for  conventional  feedback  decoders  (see  Heller  [l6l).  Thus,  for 
comparison  purposes,  as  the  three  level  data  structure  is  loaded  to 
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generate  the  SUBMAP  table,  one  can  also  generate  a fourth  level 


containing  a "minimum  weight"  look-up  table  obtained  by  associating 

with  each  Z the  error  bit  found  in  a minimum  weight  E1  mapping 

t - A 

to  this  Z.  This  was  done  in  the  creation  of  look-up  tables  for  the 
code  and  LSYN  combinations  simulated  in  Section  5.  5 to  enable  a 
side  by  side  comparison  of  actual  SUBMAP  versus  actual  conventional 
feedback  performance. 

The  same  upper  bounding  arguments  described  in  Section  5.2,2 
for  the  SUBMAP  table  can  be  applied  to  the  minimum  weight  error 
vector  table  (conventional  feedback)  to  give  an  upper  bound  on  the 
performance  of  a decoder  using  the  conventional  feedback  table 
without  feedback.  Figure  5.5  shows  the  predicted  performance 
bounds  for  both  the  SUBMAP  and  conventional  minimum  weight  tables 
for  LSYN  = 13  with  a K = 11  code  at  a constant  SNR  level  of  2 db 
(E  /N  A.  As  expected,  the  SUBMAP  upper  bound  remains  lower 
than  tin-  minimum  weight  (without  feedback)  upper  bound  as  MAX  is 
increased  to  consider  a larger  subset  of  lightweight  error  vectors. 

Both  predicted  SUBMAP  and  minimum  weight  bounds  do  not 
account  for  the  potenti.il  effect  of  feeding  back  error  estimates  to 
correct  syndromes.  These  predicted  performance  bounds  give 
an  upper  limit  on  error  probability  when  the  results  of  prior 
decisions  arc  not  considered.  One  thus  is  led  to  hope  that  employing 
feedback  may  improve  SUBMAP  performance  as  it  is  known  to 


improve  minimum  weight  performance.  The  extent  to  which  this  is 


Probability  of  Bit  Error 
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true  is  shown  in  Section  5.  5 simulation. 


5.  3 Performance  Improvement  with  Increasing  Syndrome  Length 

In  order  to  demonstrate  the  beneficial  effect  of  using  syndrome 
lengths  greater  than  one  constraint  length,  predicted  SUBMAP 
performance  for  codes  of  constraint  length  K = 5,7,  and  11  were 
computed  for  several  LSYN  values.  These  predictions  are  given 
in  Figures  5.6,  5.7,  and  5.8  respectively  and  show  how  performance 
should  improve  as  the  number  of  syndrome  bits  used  is  increased. 

From  these  predictions  one  can  also  observe  that,  for  a given 
LSYN,  substituting  a code  with  a longer  constraint  length  and  mini- 
mum distance  for  a shorter  constraint  length  code  should  provide 
better  performance.  LSYN  is  the  parameter  that  determines  the  size 
of  the  SUBMAP  look-up  table  and  thus  effectively  the  complexity  of 
the  decoder.  Thus  in  a practical  design  one  is  led  to  allow  the 
permitted  level  of  complexity  (look-up  table  size)  determine  a 
corresponding  LSYN  value  and  use  a code  having  a constraint  length 
as  near  to  this  LSYN  as  possible. 

Specifically,  using  a longer  constraint  length  code  (e.g.  I<  = 11) 
provides  better  performance  than  using  a shorter  constraint  length 
code  (e.g.  K = 5)  for  the  same  number  of  syndrome  bits  (e.g.  LSYN'll). 

This  is  to  be  expected  due  to  the  greater  minimum  distance  (d  =7) 

min 

of  the  K = ll  code  versus  d =4  for  the  K=5  code. 

min 
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Probability  of  Bit  Error 


Probability  of  Bit  Error 


L 


Probabil 


Also  plotted  in  Figures  5.  6 and  5.  7 are  extracted  Viterbi 
decoding  results  as  published  by  Heller  and  Jacobs  [ 1 7]  for  K = 5 
and  K =7  respectively.  The  side-by-side  comparison  of  Viterbi 
decoding  with  SUBMAP  decoding  indicates  the  greater  range  of 
coding  gain  achievable  by  the  more  complex  Viterbi  approach.  This 
point  will  be  touched  on  again  in  Section  5.4  where  a loose  lower 
bound  on  SUBMAP  performance  (for  a fixed  decoding  delay  A)  will  be 
seen  to  lie  above  the  published  Viterbi  performance. 

5.  4 A Loose  Lower  Bound  on  Probability  of  Error 

The  previous  sections  have  shown  that  for  a given  code  and 
delay  A one  can  achieve  improved  SUBMAP  performance  by  using 
longer  syndrome  vectors  as  shown  in  Figures  5.  6 through  5.  8. 

It  would  be  useful,  when  comparing  the  suboptimal  SUBMAP  decoder 
performance  with  other  decoder  approaches  to  have  some  lowe r 
bound  on  SUBMAP  probability  of  error.  This  would  enable  one  to 
state  a performance  improvement  limit  which  even  continued 
effort  at  increasing  the  SUBMAP  look-up  table  size  would  not 
achieve.  A computational  method  of  specifying  such  a lower  bound 
for  a given  delay  and  code  can  be  obtained  by  examining  the 
syndrome  equation  set.  If  one  hypothesizes  that  no  prior  incorrect 
error  estimates  have  been  made  and  that  all  prior  estimates  have 
been  fedback,  one  can  examine  the  resultant  first  error  probability 
to  obtain  a bound. 
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As  an  illustration,  consider  the  previously  discussed  K = 5,  rate  ' 

systematic  code  for  which  the  first  six  syndrome  equations  are: 

..  (2)  (11  (1)  (11  - (1) 

Zt-5  = 't-5+et-5+et-6+Ct-7+0+'t-9 

Z „ - e12'  le"’  e(1»  +e"'  + 0te,"> 

t-4  t-4  t-4  t-5  t-6  t-8 

-/  x x (!)  . (1)  . n , 

t-3  t -3  t - 3 ■ t-4  t-5  t-7 

(2)  (1)  (1)  (1)  „ ( 1 1 
Z ^=el+e'+e'+e'+0+e  ' 

t-2  t -2  t -2  t-3  t-4  t-6 

Z = e(2)  +e(1)  +e(1)  +e(1)  +0+e(1) 

t-1  t-1  t-1  t-2  + t-3  t-5 


(2)  (1)  (1)  (1)  (1) 

= e + e + e + e + 0 + e . 

t t t-1  t-2  t-4 


Here  with  ^ = 5 the  decoder  is  now  to  estimate  e 


(5.231 


If  one  now  considers  the  conditional  probability  of  the  decoder 
making  a mistake  in  estimating  conditioned  on  all  prior  estimate: 

being  both  correct  and  correctly  fedback.  the  following  set  of 
modified  syndrome  equations  apply. 

Z'  =e(2)+e(1) 
t-5  t-5  t-5 

•7.  - <2'  x (1)  X 

Zt_4  et-4  + et-4  + Ct-5 

71  J2)  X Jl)  4 J1]  X J1' 

Zt-3  - et-3  +Ct-3  +Ct-4  +<?t-5 

v,  ■ -El 


(2)  (1)  (1)  (11  . „ . (M 

= 't  +Ct  +et-l  ,e,-2+0+Ct-4 


(5.24) 


Here  the  upper  right  triangle  of  error  bits  in  (5.23)  has  been  re- 
moved and  the  associated  syndrome  bits  modified  to  corrected  values 
to  reflect  the  error  bit  removal  through  feedback. 

Faced  with  the  modified  set  of  syndrome  equations  for  this 
idealized  conditional  case,  one  can  compute  a ROMMAP  or  SUBMAP 
look-up  table  as  previously  discussed.  Then  the  conditional  prob- 
ability of  incorrectly  estimating  ej  ^ can  be  computed  exactly  in  the 
ROMMAP  case  or  bounded  in  the  SUBMAP  case  as  explained  in 
Sections  5.1.3  and  5.2.2  respectively.  In  the  ROMMAP  case  this 
idealized  probability  of  error  will  be  a lower  bound  on  ROMMAP 
performance  no  matter  how  man y additional  syndrome  bits  might  be 
used.  In  the  SUBMAP  case  the  idealized  probability  will  be  a tight 
upper  bound  on  the  P^  and  being  a tight  upper  bound  for  such  an 
idealized  case  will  be  useful  as  a loose  lower  bound  to  actual 
performance . 

This  SUBMAP  lower  bound  has  been  computed  for  the  K=5 
(A  = 5)  and  K=ll  (A  =10)  decoders  and  also  plotted  on  Figure  5.6  and 
5.  8 respectively.  Since  the  assumption  of  no  previous  incorrect 
error  estimates  is  quite  idealized,  this  first  error  probability  lower 
bound  is  relatively  loose.  Indeed  as  shown  in  later  simulation 
results  for  the  I<=5  code  in  Figure  5.9,  continued  increases  in 
syndrome  length  with  a fixed  delay  causes  performance  to  tend 
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toward  an  improvement  limit  si  db  above  t he  loose  lower  bound. 


5.  5 Simulation  Results 

In  this  section  performance  curves  from  Monte  Carlo  simula- 
tion for  a number  of  SUBMAP  decoder  implementations  are  pre- 
sented. Following  some  preliminary  explanation  regarding  simula- 
tion methods,  actual  SUBMAP  performance  results  for  two  system 
atic  codes  listed  by  Bussgang  [lo’1  for  constraint  lengths  of  I<=5 
and  K=1 1 are  given.  Although  primary  emphasis  is  placed  on 
SUBMAP  decoder  performance,  some  cases  were  simulated  in 
which  the  conventional  feedback  decoding  approach  (using  minimum 
weight  error  vectors)  as  well  as  the  SUBMAP  decoder  were  run 
with  the  same  error  inputs  to  enable  a side-by-side  comparison  of 
the  two  methods.  The  performance  curves  show  the  improvement 
possible  not  only  by  using  longer  constraint  length  codes  but  also 
that  by  using  more  syndrome  bits  in  the  SUBMAP  decoding 
of  a particular  code.  Also  displayed  is  the  extent  of  performance 
improvement  achievable  with  no  complexity  increase  by  merely 
substituting  the  SUBMAP  table  for  the  conventional  feedback  look  up 
tables . 

5.5.1  Method  of  Simulation 

The  simulation  described  in  the  following  section  was  done  in 
FORTRAN  on  a Digital  Equipment  Corporation  KL  10  time -sharing 
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system  (in  batch  mode)  located  at  the  University  of  Southern 
California.  All  codes  considered  were  optimum  rate  z systematic 
codes  given  in  Bussgang  [19]  . Two  uniform  pseudo-random  number 
generators  (PRN)  were  used  interchangeably  at  critical  points  to 
check  the  quality  of  the  error  generation  process.  Appendix  B gives 
additional  details  on  these  PRN  generators.  To  insure  the  statistical 
validity  of  simulation  results,  error  bit  inputs  to  the  two  syndrome 
former  input  ports  were: 

(1)  independently  generated  through  separate  calls  to  the  PRN 
generator  using  seeds  selected  to  guarantee  no  repetition 
during  10x10^  calls  to  the  PRN  generator;  and 

(2)  simulated  for  at  least  10  error  bits  per  syndrome  former 
input  except  at  higher  SNR  levels  were  up  to  5x1  (/’  error  bits 
per  input  were  simulated.  All  P^  points  plotted  herein  are 
based  on  at  least  100  decoding  errors. 

The  general  operation  of  the  simulation  program  consisted 
of  reading  in  and  storing  the  appropriate  look-up  table  previously 
generated  in  accordance  with  Section  5.2  . This  look-up  table  would 
have  been  created  as  cither  a SUBMAP  table  or  as  a minimum 
weight  table  corresponding  to  the  decoder  type  being  simulated.  The 
PRN  generator  then  supplied  two  PRNs  uniformly  distributed  between 
0 and  1.  These  were  used  to  create  e.  and  e.  ' error  bit  inputs 

i i r 
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based  on  the  HSC  crossover  probability  given  by  the  symbol  SNR 


(E^/Nq)  as  specified  in  Section  5.  1.2.  The  syndrome  former  then 

computed  the  syndrome  bit  7.  for  input  to  the  syndrome  vector  while 

error  bits  were  shifted  along  each  clock  period  in  an  E vector. 

These  two  error  bits  were  actually  input  to  two  parallel  decoders, 

one  decoding  based  strictly  on  the  current  with  no  feedback  of 

error  estimates  and  a second  using  feedback  of  error  decisions  to 

update  the  syndrome  vector.  Thus  actual  performance  for  a 

particular  look-up  table  could  be  simulated  in  one  pass  both  with 

and  without  feedback.  The  actual  P for  both  decoders  was 

e 

computed  by  counting  the  number  of  times  during  the  simulation  that 

A (1 ) (J  \ 

the  estimated  error  e.  did  not  equal  the  actual  error  bit  e' 

i-A  i-A 

and  dividing  by  the  number  of  error  bits  processed  during  the 
simulation  through  one  input  to  the  syndrome  former. 

Since  the  operation  of  the  syndrome  forme’-  is  independent  of 
the  actual  data  that  might  have  been  input  to  the  encoder,  the 
above  simulation  effectively  assumes  all  zero  data  and  thus  only 
error  inputs  entering  the  syndrome  former. 

The  PRN  generators  used  were  such  that,  given  a starting 
"seed"  number,  the  generator  would  output  the  identical  stream  of 
PRNs.  Thus  it  was  possible  to  run  simulations  of  various  SUh MAP 
and  conventional  feedback  schemes  vising  the  identical  input  error  bit 
streams.  This  gives  a more  reasonable  comparison  of  techniques 

14.1 


than  non-idcntical  input  streams. 

5.  5.  2 Decoder  Performance  for  K-5  Code 

The  decoding  of  the  K-5  binary  systematic  rate  J code  (Gl  = 
10000,  G2  - 11101)  used  in  previous  performance  prediction  dis- 
cussions was  simulated  for  both  the  SUBMAP  and  the  conventional 
feedback  approaches.  As  described  in  Section  5.5.  1,  the  perform- 
ance of  each  decoder  table  was  simulated  both  with  and  without 
feedback.  The  decoding  delay  was  fixed  at  A = 5 (i.e.  estimate 
ef  j.)  with  MAX  = 5 in  the  table  creation  step.  The  design  SNR  was 
always  2 db  (E  /N  ). 

•5  U 

Figure  5.9  shows  simulation  results  for  the  K=5  code  SUBMAP 
decoder  with  and  without  feedback  for  LSYN  = 6 and  LSYN  = 11. 

Also  simulated  but  not  shown  was  LSYN  = 15.  Figure  5.9  also  gives 
computed  upper  bounds  and  the  loose  lower  bound  for  A =5.  It  can 
be  seen  that  the  simulated  performance  falls  extremely  close  to  the 
predicted  tight  upper  bounds  and  well  above  the  loose  lower  bound. 
The  futility  of  increasing  LSYN  much  beyond  two  constraint  lengths 
is  actually  indicated  by  what  is  not  plotted  since  both  predicted 
performance  and  simulation  results  for  LSYN  = 15  fell  so  close  to 
LSYN  = 11  bound  and  results  so  as  to  make  plotting  them  infeasible. 

Also  evident  is  the  minimal  improvement  achievable  by  vising 
feedback  with  the  SUBMAP  decoder.  Indeed  using  feedback  in  the 
LSYN  = 11  case  produced  somewhat  poorer  simulated  performance. 
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Figure  5.  10  shows  simulation  results  for  a minimum  weight 
look  -up  table  decoder  with  and  without  feedback  for  the  same  K=5 
code  for  LSYN=6  and  LSYN=11.  The  improvement  due  to  feedback 
is  seen  to  be  more  substantial  for  the  minimum  weight  case  than  for 
the  SUBMAP  case  at  LSYN=6.  However  at  LSYN  = 11  feedback  appears 
to  be  counterproductive  since  feedback  results  for  the  minimum 
weight  approach  practically  coincide  for  both  LSYN  = 6 and  LSYN  =11. 

Figure  5.11  combines  selected  SUBMAP  and  conventional 
feedback  results  on  one  graph  to  show  the  relative  superiority  of  the 
SUBMAP  approach  over  the  minimum  weight  approach.  A careful 
comparison  indicates  that  the  SUBMAP  approach  even  without  feed- 
back slightly  outperforms  conventional  feedback  decoding.  However 
the  absolute  performance  improvement  for  the  short  (K  = 5)  constraint 
length  code  and  LSYN  combinations  considered  was  roughly  0.2  dB 
at  lower  SNR  levels. 

In  summary,  although  these  simulation  results  confirm  the 
accuracy  of  the  predicted  SUBMAP  performance  bounds,  they  reveal 
only  a minimal  improvement  over  conventional  feedback  decoder 
performance.  Therefore  the  following  section  deals  with  SUBMAP 
performance  for  a longer  constraint  length  ( K = 1 1 ) code. 

5.  5.  3 Decoder  Performance  for  K=ll  Code 

The  SUBMAP  decoding  of  the  K=ll  systematic  rate  code 
specified  by 
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(5.25) 


G2  = 11011100100 

was  simulated  both  with  and  without  feedback  for  LSYN  = 11  and 
LSYN  = 13.  A design  SNR  of  2 db  (F  /N^)  was  used  witfi  MAX=5 
for  look-up  table  creation.  The  decoding  delay  was  fixed  at  A = 10 
(i.e.  estimate  e^|„).  Also  simulated  was  decoding  using  a mini- 

i = 10 

mum  weight  approach  as  used  in  conventional  feedback. 

Figure  5.12  shows  SUBMAP  simulation  results  for  the  K=ll 
code  for  LSYN  = 11  and  LSYN  = 13.  Evident  is  a slight  perform- 
ance improvement  obtained  by  using  feedback  with  the  SUB  MAP 
table  at  both  LSYN  values.  Shown  as  solid  lines  are  the  predicted 
upper  bounds  and  the  loose  lower  bound.  Again  the  accuracy  of 
the  upper  bounds  is  confirmed. 

Figure  5.13  shows  simulation  results  using  a minimum  weight 
look-up  table  to  demonstrate  conventional  feedback  performance  for 
the  K = 11  code  at  LSYN  =11  and  13,  Shown  as  solid  lines  are  the 
predicted  upper  bounds  and  the  loose  lower  bound.  Evident  is  the 
performance  improvement  obtained  by  using  feedback. 

Figure  5.14  combines  SUBMAP  and  conventional  feedback 
results  for  LSYN  = 1 1 on  one  graph  to  show  the  relative  superiority 
of  the  SUBMAP  approach.  Thus  is  evident  the  performance 
improvement  possible  with  no  decoding  complexity  increase  by 
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merely  substituting  a SUBMAP  look-up  table  for  the  minimum 
weight  table  in  a conventional  feedback  decoder. 

The  statement  was  made  in  Section  5.  3 that  it  is  better  to  use 
a longer  constraint  length  code  rather  than  merely  increasing  LSYN 
for  a shorter  constraint  length  code.  This  fact  is  shown  to  be  true 
in  Figure  5.  15  where  simulation  results  for  K=5  (LSYN  = 1 1)  and 
K = 11  (LSYN  =11)  have  been  compared.  It  is  evident  that  the  K=ll 
code  is  preferable  if  hardware  complexity  equivalent  to  a size 
2 NS'S  N _ 211  ,g  feaSjbie-  The  improvement  is  to  be  expected  since 
the  minimum  distance  of  the  K=5  code  was  4 while  that  of  the  K=ll 
code  was  7.  The  additional  error  correction  capability  exhibits 
itself  in  the  steeper  slope  to  the  K=1  1 curve  of  P . 
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CONCLUSIONS  AND  RECOMMENDATIONS 

This  investigation  developed  in  Chapter  3 several  optimal  MAP 
error  bit  estimation  algorithms  for  decoding  convolutional  codes 
transmitted  over  a binary  symmetric  channel.  These  represent  a 
set  of  error  decoding  approaches  complementary  to  similar  tech- 
niques previously  used  for  data  estimation.  One  advantage  of  such 
error  bit  estimation  using  syndromes  is  that  the  resulting  perform- 
ance is  independent  of  a priori  data  probability.  Thus  an  arbitrary 
actual  data  source  output  can  be  communicated  reliably- without 
having  to  preprocess  the  data  prior  to  encoding  jus!  to  insure 
equilikely  input  bits.  This  is  in  contrast  to  standard  Viterbi  maxi- 
mum likelihood  sequence  decoding  which  requires  equilikely  input 
probabilities.  Also  in  Chapter  3 corresponding  optimal  MAP  error 
algorithms  have  been  derived  to  handle  the  more  realistic  case  of 
error  decoding  under  a real  time  or  finite  delay  restriction. 

During  the  Chapter  4 investigation  of  implementable  ROMMAP 

and  SUI3MAP  suboptimal  decoders  vising  MAP  error  bit  estimation 

with  finite  delay,  it  was  seen  that  conventional  feedback  decoding  is 

a rough  approximation  to  MAP  error  bit  estimation.  The  results 

of  simulation  discussed  in  Chapter  5 showed  that  a SUBMAP  decoder 

based  on  MAP  estimation  outperforms  conventional  feedback  decoding 
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for  the  same  complexity.  Noted  also  was  the  fact  that  feeding  back 


error  decisions  in  the  SUBMAP  case  does  provide  additional  improve- 
ment. 

The  complexity  parameter  for  ROMMAP,  SUBMAP,  and  con- 
ventional feedback  is  basically  the  look-up  table  size  (2  ).  Thus, 

based  on  the  simulation  results  of  Chapter  5,  for  a given  complexity 
one  is  inclined  to  favor  use  of  a SUBMAP  look-up  table  instead  of  a 
minimum  weight  look-up  table  as  used  in  conventional  feedback 
decoders.  One  other  point  worth  mentioning  in  this  connection  is 
the  comparative  "weights"  of  the  look-up  tables,  that  is  the  number 
of  l's  in  the  table  entries.  In  all  cases  considered  here,  the  SUBMAP 
table  always  had  significantly  less  l's  than  did  the  minimum  weight 
table  of  the  same  2^^^  size.  For  example  Table  6.  1 shows 
comparative  weights  for  the  K = 11  systematic  code  at  LSYN  =11 
and  13.  This  investigation  assumed  and  simulated  read  only  memory 
(ROM)  look-up  tables  in  which  the  syndrome  vector  Z was  converted 
to  an  address  in  a 2^^^^-size  table  where  each  address  location 
contains  either  a 1 or  0.  However  if  an  alternative  device  such  as 
one  based  on  an  associative  memory  were  used,  the  SUBMAP  table 
implementation  would  be  much  simpler  than  that  of  the  minimum 
weight  table.  This  is  due  to  the  smaller  number  of  1 -associated 
syndromes  7*_  (i.  e.  the  lighter-weight  table)  of  the  SUBMAP  approach 
which  would  require  a significantly  smaller  associative  memory 
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Table  6.  !.  Look-up  Table  Weights  for  K 11  Code  Example. 


device.  This  appears  to  be  an  area  worthy  of  some  additional  investi- 
gation in  light  of  continuing  advances  in  LSI  capability. 

The  simulation  results  of  Chapter  5 also  demonstrated  the  accu- 
racy of  the  tight  upper  bounds  on  performance  derived  in  Section  3.  2.  2 
thereby  implying  that  these  bounds  represent  a valuable  design  tool 
for  predicting  actual  decoder  performance.  Section  5.  3 confirmed 
the  expected  improvement  in  Sl'llMAP  decoder  performance  with 
increasing  syndrome  vector  length.  This  section  also  validated  the 
rule  of  thumb  that,  with  a complexity  limit  on  syndrome  vector  length, 
the  SUI3MAP  designer  should  select  a code  having  as  long  a constraint 
length  as  possible  within  this  syndrome  length  for  best  SPI3MAP 
pe  rfo  rma  nee. 


1ST 


The  practical  hardware  limits  to  increasing  look-up  table  size 


is  obviously  a major  limitation  on  continued  SUBMAP  performance 
improvement.  Thus  it  is  fell  that  further  development  of  the  tech- 
niques derived  in  Chapter  3 for  optimal  error  bit  estimation  could 
lead  to  an  implementation  scheme  that  would  bridge  the  gap  between 
the  performance  of  the  look-up  table  approach  of  Sl'BMAP(and  indeed 
of  conventional  feedback)  and  that  of  Viterbi  maximum  likelihood 
sequence  decoding.  Bridging  this  gap  with  such  a MAP  error  bit 
estimation  approach  based  on  syndromes  would  enable  decoder 
performance  to  be  improved  yet  still  maintain  performance  independ- 
ent of  a priori  input  data  probability. 
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APPENDIX  A 


VITERBI  DECODING  WITH 
NONEQUILIKELY  INPUT  SEQUENCES 


As  stated  by  Viterbi  [ 14], 


. , . for  completely  general  channels.  . . if  all  input  data 
are  equally  likely,  the  decoder  which  minimizes  the 
error  probability  is  one  which  compares  the  condition- 
al probabilities,  also  called  likelihood  functions, 
P(_y/x(‘  5)  where^  is  the  overall  received  sequence  and 
x*  is  one  of  the  possible  transmitted  sequences,  and 
decides  in  favor  of  the  maximum.  This  is  called  a 
maximum  likelihood  decoder.  The  likelihood  functions 
are  given  or  computed  from  the  specifications  of  the 
channel.  Generally  it  is  more  convenient  to  compare 
the  quantities  logP(^/jc(l '),  called  log  likelihood  func- 
tions and  the  result  is  unaltered  since  the  logarithm  is 
a monotonic  function  of  its  (always  positive)  argument. 


The  important  point  for  the  reader  to  bear  in  mind  is  that  Viterbi' s 
excellent  description  of  his  now  famous  algorithm  applies  only  to  the 
case  of  equally  likely  input  sequences. 

To  analyze  the  case  where  the  input  data  sequences  arc  not  equi- 
likely,  a maximum  a posteriori  decoder  should  be  considered.  This 
is  one  which  compares  the  conditional  probabilities 


P(  x(m)/_y)  = 


P(i/x(m))P(x(m)) 

P(I) 


(A.  1) 


/ \ 

for  candidate  sequences  fx  ' } and  decides  in  favor  of  the  maximum. 

Now  consider  the  specific  case  of  Section  3.  2 where  a rate  1/3 
decoding  problem  has  been  converted  to  a rate  1/2  encoder  and 
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channel  with  error  bit  sequence  inputs.  This  provides  a good  ex- 
ample of  what  adaptation  is  necessary  for  a Viterbi  algorithm  for  a 
nonequilikely  input  case.  Thus  Figure  3.  3.b  is  the  "error  encoder" 
that  will  now  be  analyzed.  Here  define  P(e|^=  1 ) = p and  P(e^  = 0)  = 
(1-p)  since  the  binary  symmetric  channel  still  applies.  In  this  case 
Eq.  A.  1 becomes 


maximize 
over  a 1 1 cj_E) 


p(c(E)/Z) 


p(_Z/c(E)P(c(E))  ) 


F(X) 


i 


(A.  2) 


where  to  obtain  A.  2 the  following  substitutions  have  been  made: 


(1)  becomes  = the  vector  of  received  syndrome  bits; 

(2)  becomes  c(E)  - one  of  the  possible  "transmitted" 
sequences;  and 

A 

(3)  _E  = (Oq,  e j , . . . , e^)  = input  error  vector. 

Note  that  each  possible  input  error  vector  _E  corresponds  to  only 
one  _c(_E)  just  as  a data  encoder  output  maps  directly  back  to  one  data 
input  sequence  in  the  more  conventional  data  encoding.  Due  to  the 
bit-to-bit  independence  of  all  error  bits,  A.  2 can  be  rewritten  as: 


maximize  over 
all  c(E) 


N-l 

F>(c(_E)/X.)=  n p(Z./c.(E))P(c.(E)) 

! - A ^ t l 


(A.  3) 


The  elements  in  Eq.  A.  3 are: 


lbO 


r 


i 


(1)  Z; 


z(2) 


z<3> 


L 1 J 


'received"  syndrome  bits  at  time  i; 


r c(2)  1 

1 


(2)  c.(E)  = 


c<3) 

i 


= "transmitted"  bits  at  time  i;  and 


(3)  N = length  of  the  input  error  sequence. 


Taking  logarithms  yields: 


maximize 
over  all  c(  E) 


N-l 


logP(c(E)/Z)=  Z log(P(Z./c  (_E))P(c  (E))) 
i = 0 1 


(A.  4) 


where  the  sum  is  over  all  branches  in  the  error  encoder  trellis  path 


corresponding  to  this  particular  IC  vector 

A 


Now  define  Il(a,  b)  — Hamming  distance  between  vectors  and  _b. 
Also  note  that  the  P(c.(E))  is  just  the  a priori  probability  of  the  input 


(1) 


error  bit  e.  ; for  the  c(E)  in  question  (either  p or  1-p  since  the  "in- 


puts" are  characterized  by  the  binary  symmetric  channel  crossover 
probability  and  there  is  a one-to-one  mapping  from  inputs  to  outputs). 
Hence, 


maximize 
over 


allc(E)|lo*^>/^ 

N-l 


H(Z.,c  (E))  n -H(Z  , c (E)) 

= E |08(P  11-p)  r<=!  ')>l 

1 = 0 (A. 5) 


maximize 
over  all  c(  E) 


N-1  (n 

Z [H(Z.,c.(E))logp+(ti0-H(Z.,c(E))log(l-p)  + logP(e|  ')] 
i = 0 

(A.S) 
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where  = the  number  of  bits  in  and  cT_E)  (r>g=  2 in  this  specific 
case). 

Rearranging  yields: 

constant 

N-l  , 

53  [-H(Z.,  c.(E))log  — +nQlog(l-p) 

l=°  I 

+ log  P( e. )]  ) . (A.  6) 

It  should  be  noted  that  the  third  term  of  the  summation  will  equal 
log(p)  or  log(l-p)  depending  on  the  value  of  e|^  specified  by  the  er- 
ror input  vector  _E  being  considered.  Also,  if  P(ef  l)  = P(e!  ^-0)  = 

f (as  would  be  true  only  if  the  error  inputs  were  equilikely),  Eq. 

A.  6 would  reduce  to  the  conventional  Viterbi  decoding  rule: 


maximize 
over  all  c_(E)) 


mT;Tl\Zl(E)  % [-H(Zi,Ci(E))Ki  + K2] 


ovei 


/ith  Kj  = log^-^-1^  and  I<2  = nQlog(  1 -p)  + log(  f). 


(A. 7) 


This  would  direct  the  decoder  to  select  the  path  cj_E)  minimizing  the 
Hamming  distance  between  _Z  and  _c(JC). 

On  the  contrary,  for  the  nonequilikely  input  case  under  consider- 
ation here,  Pfe*.1^  1)  t Pte^^O).  Thus  log  P(e[1))  must  be  included 
in  all  computations  and  a modified  metric  must  be  used.  Generally 
A.  6 requires  that  for  a maximum,  the  trellis  path  would  be  selected 
which  both: 


(1)  minimizes  H(_Z,c(E))  weighted  by  log^-^-r^  and 
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(2)  maximizes  log  Pf^E)  = logP(e.  ). 

i = 0 1 


The  quantity  logP(e'^  ) is  either  log(p)  or  log(l-p).  Since  0 ?p? 

1 and  0^  l-p«  1,  this  logarithm  is  nonpositive.  The  least  negative 
logarithm  corresponds  to  ef^  = 0.  Thus  the  maximization  in  A.  6 im- 
plies that  in  selecting  _E  one  should  "lean"  toward  an  input  sequence 
with  minimum  weight  whose  output  sequence  c_(_E)  is  closest  to  the 
actual  received  syndrome  sequence  _Z. 

With  the  definition  H(_E,  0)  — Hamming  distance  between  a candi- 
date E and  the  all-zeros  vector,  A.  6 can  be  expressed  entirely  in 
terms  of  Hamming  distances.  Neglecting  constant  terms,  the  A.  6 
rule  can  be  written  in  an  equivalent  form  as: 


maximize 
ove r all  c ( E) 


pH(_Z,c(E))pH(E,0) 

(1.p)H(Z,c(E))(1_p)H(E,0) 


(A. 7) 


maximize  I.  7 /1-p  \-[H(  Z,  c(  E))+H(  E,  0)] 
over  all  _c(_E)  | °^  \ p ) 


"*eXr”iri(E){-|H|Z-  '<  E»+H(  E’  °>1  1°8(tE) 

constant 


minimize 


Optimum  decoding 

j^'e  [H(  Z,  c(  E))  -t  H(  E,  0)1  rule  for  nonequilikely 

' i n r>i  l t c 


inputs  . 


(A. 8) 
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Thus  the  decoding  rule  for  error  sequences  in  this  case  requires 
the  selection  of  _E  as  that  input  error  sequence  which  minimizes  the 
Hamming  distance  sum  given  by  A.  8. 

Since  the  Hamming  distance  H(_E,  0)  is  the  same  as  the  Hamming 
weight  of  the  error  vector  _E,  the  decoding  rule  for  nonenuilikely  in- 
put sequences  can  also  be  written  as: 


minimize 
over  all  c_(_E) 


[H(Z,  c(E))  + W(E)] 


(A.  9) 


where  W(E)  is  the  weight  of  the  error  vector  E. 
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APPENDIX  13 

COMPUTER  PROGRAMMING  CONSIDERATIONS 

The  ROMMAP  and  SUBMAP  creation  programs  for  the  K = 5 and 
K = 7 codes  discussed  in  Section  5.  1 and  5.  2 were  initially  pro- 
grammed on  an  IBM  360/44  system  in  FORTRAN  IV.  The  necessary 

conversion  from  SNR  (E  /N  ) to  BSC  crossover  probability  p was 

so  r 

done  as  explained  in  Section  5.  1.  2 using  the  double  precision  mathe- 
matical function  DERFC(x)  [30]. 

Due  to  360  core  limitations  and  operating  system  reliability,  the 
maximum  syndrome  length  for  which  SUBMAP  look-up  tables  could 
be  created  using  the  360  was  approximately  LSYN=  13  (requiring 
about  2 hours  of  CPU  time).  Consequently  the  SUBMAP  creation  and 
performance  bounding  programs  were  converted  to  the  KL-10  Digi- 
tal Equipment  Corporation  system  using  FORTRAN-10  [3l].  Since 
FORTRAN-10  provides  a very  limited  selection  of  mathematical 
functions,  the  SNR-to-p  conversion  was  implemented  using  a called 
subroutine  MERFC  from  the  IMSL  library  of  subroutines  [32].  A 
comparison  of  both  ERFC  functions  was  made  to  verify  that  no  signi- 
ficant differences  existed  over  the  range  of  SNR  investigated  in  this 
work. 

The  simulation  programs  described  in  Section  5.5  were  all  run 
on  the  KL-10  system  which  could  practically  support  syndrome 
lengths  of  LYSN  = 17  (equivalent  to  roughly  4 hours  of  CPU  time). 
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Figure  5.13  Minimum  Weight  Simulation,  K = 11 
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Since  simulation  results  could  be  sensitive  to  the  quality  of  the 
pseudo- random  number  (PRN)  generator  used,  two  different  genera- 
tors were  implemented  and  compared.  These  were: 

(1)  GGUB,  a standard  subroutine  from  the  IMSL  library  [32] 
and  (2)  REGOL,  a PRN  subroutine  coded  at  USC. 

It  was  found  that  for  the  SNR  range  investigated  (0  to  5 db),  no  appre- 
ciable difference  in  simulation  results  was  obtained.  Figure  B.  1 
shows  comparison  results  for  REGOL  and  GGUB  for  a typical  rate 
1/2  simulation  with  10*  calls  for  each  of  the  two  error  inputs.  It  can 
be  seen  that  the  simulation  results  for  both  PRN's  coincide  except  at 
the  higher  SNR  level  of  5 db  Eg/No.  Thus  in  practice,  higher  SNR 
levels  (e.  g.  , 4 and  5 db)  were  simulated  using  greater  than  ic/  calls 
per  input  always  insuring  100  or  more  actua'  errors  were  created. 

For  reference  purposes  a brief  description  of  the  REGOL  sub- 
routine as  implemented  by  G.  G.  Finn  and  J.  M.  Smith  of  USC  follows. 
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FUNC  TION  REGOL 

AUTHORS 

G.  G.  FINN,  IMAGE  PROCESS.  INST. 

J.M.  SMITH,  DEPT.  ELEC.  ENG. 

UNIV.  SO.  CALIF. 

PURPOSE 

GENERATES  A STANDARD  UNIFORMLY  DISTRIBUTED 

RANDOM  NUMBER  (UNIFORM  ON  (0,  1)). 

USAGE 

X = REGOL(IR) 

DESCRIPTION  OF  PARAMETERS 

IR  - INPUT.  FOR  THE  FIRST  ENTRY  THIS  VARIABLE 

MUST  BE  INITIALIZED  TO  ANY  ODD  INTEGER  IN 
THE  EXCLUSIVE  RANGE  (0,34359738368).  UPON 
RETURN  IT  WILL  CONTAIN  A NEW  ODD  INTEGER 
TO  BE  USED  ON  THE  NEXT  ENTRY  TO  THE  SUB- 
PROGRAM. THIS  VARIABLE  MUST  NEVER  BE 
REDEFINED  BY  THE  CALLING  PROGRAM; 
OTHERWISE  THE  SEQUENCE  OF  RANDOM  NUMBERS 
WILL  BE  DISTURBED. 

REGOL  - OUTPUT.  STANDARD  UNIFORMLY  DISTRIBUTED 
REAL  RANDOM  NUMBER. 

REMARKS 

THIS  SUBPROGRAM  IS  SPECIFIC  TO  THE  DECSYSTEM-10 

AND  DEC  SYSTEM- 20  SERIES  OF  COMPUTERS. 

METHOD 

THIS  SUBPROGRAM  REALIZES  THE  MULTIPLICATIVE 

CONGRUENTLAL  METHOD  (PERIOD  2**33)  USING  A MULTI- 
PLIER CHOSEN  ACCORDING  TO  THE  RECOMMENDA  TION  OF 

REFERENCE  1. 

REFERENCE 

[1]  J.H.  AHRENS,  U.  DIETER,  AND  A.  GRUBE,  COMPUTING, 
VOL.  6,  PP.  121  -138,  1970. 
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based  on  MAP  estimation  outperforms  conventional  feedback  decoding 
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APPENDIX  C 

OPTIMAL  FINITE  DELAY  DECODING  WHEN  A s v 


In  the  optimal  decoding  algorithm  when  J s \j  as  noted  in  Chapter 
3 Section  3.  3, there  is  a branch  from  Step  1 to  Step  4 given  below. 


Step  4:  (Final  average  and  decision) 
For  i = 0,  1 calculate 


V £ •••  E q(et-v=i’e(t-v+i.t)) 

et-vtl  Ct 

If  An  > ‘e'l  = 0,  otherwise  e = 1. 
0 1 t-v  t-v 

Continue  with  Step  5. 


For  illustration,  consider  t = v + 1-  Step  4 yields 


A0  E ^ ^el  ®’°(2,  v + l)’  J tree  with  e,  = 0,  i.  e.  , the 

e2  ev+l 


. _ / sum  of  alt  paths  in  error 
’ ~ i i m - 

1 

lower  half  tree. 


^1  E E e j ' ’ e(2,  v + 1)’  J tree  with  e.=  l,  i.  e. , the 


'v  + 1 


. _ / sum  of  all  paths  in  error’ 
'“It  t u nr-’  - _ 

1 

upper  half  tree. 


But  there  is  a number  for  each  q(e^  v+1)^  error  tree- 

Thus  Step  4 merely  requires  the  adding  together  of  the  q(e^  ^ + 
at  the  end  of  the  appropriate  half  trees  to  make  the  decision  on  cj  ^ 
(in  this  case  on'e'j). 


For  completeness.  Step  5 following  Step  4 is  given  below. 


Step  5:  (following  Step  4 for  case  of  ^ s y). 

Increase  t by  I. 

Calculate 

q(e(t-v.  t)*  = P(et)P(Zt/e(t-v,  t)1  £ q(e(t-v-l,  t-l)) 

Ct-v-l 

Return  to  Step  1. 

Step  5 would  cause  t = v + 1 to  become  t = v+2  and 


q(e(2,  v+2))  = P(ev+2)P(Zv+2/e(2,  V+2)J  £ q(e 
e-l 

2^  ^ values 


(1,  v + 1)' 


in  this  set 


(average  over  e^) 


P(ev+2)P(Zv+2/e(2,  v+2)} 


q(er0'e(2,v+i))P(er0)+ 


q(er1,e(2,v+1))P(er1) 


Here  one  branch  in  the  upper  half  tree  has  been  added  to  the  corres- 
ponding path  in  the  lower  half  tree  with  each  path  weighted  by  P(ej  = 0) 
or  P(e.=  1).  This  results  in  a new  tree  as  shown  in  Figure  C.  1. 


Step  1:  (following  Step  5 when  ^ s v) 

Set  P(e  , Z , e,  .)  = q(e  ) 

t - V ( * # W ( t - V>  W 

Go  to  Step  4 to  continue. 
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Here,  with  t = v+Z  now,  one  would  have: 


P(eZ'Z(l,v+2)’e(3,v+2))  ^ q(e(Z,v+Z)> 


v + 1 . 

2 values 


,v  + l , 

2 values 


and  the  algorithm  would  continue  by  going  to  Step  • 


1 


1 again. 
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